The commuting schemes of an algebraic group or a Lie algebra, along with their derived versions, play a role in many areas of mathematics. They can be viewed as derived representation schemes, whose basic algebraic invariant is representation homology.The case of reductive groups and reductive Lie algebras have been studied in the literature. In the present thesis, we focus on representation homology with coefficients in non-reductive groups and non-reductive Lie algebras. We begin by establishing a connection between the commuting schemes $C(U_n)$ of a group scheme $U_n$ consisting of upper triangular unipotent matrices, and the representation homology $\mathrm{HR}_*(T^2,U_n)$ of the topological torus $T^2$ with coefficients in the group $U_n$. As an outcome, we provide a criterion for determining whether commuting schemes $C(U_n)$ form a complete intersection. We then obtain comparison theorems between different types of representation homology, and we establish an equivalence between the derived commuting scheme of a unipotent algebraic group and that of its associated nilpotent Lie algebra. Additionally, we refine the test for the formal smoothness of associative algebras via representation homology, proposed by Berest, Felder, and Ramadoss in "Derived representation schemes and noncommutative geometry" (In Expository lectures on representation theory, volume 607 of Contemp. Math., pages 113–162. Amer. Math. Soc., Providence, RI, 2014). Using the characteristic trace map linking the classical Lie algebra homology, we describe the higher-degree classes appearing in the representation homology of $2$-dimensional Lie algebras with nilpotent coefficients. This result reveals connections with classical Bott--Kostant Theorem concerning the cohomology of maximal nilpotent subalgebra of a semisimple Lie algebra.