[RnMomof7]

      7. If thou doest well, shalt thou not be accepted?--A better rendering is, "Shalt thou not have the excellency"? which is the true sense of the words referring to the high privileges and authority belonging to the first-born in patriarchal times.
      sin lieth at the door--sin, that is, a sin offering--a common meaning of the word in Scripture (as in Hsa 4:8 2Cr 5:21 Hbr 9:28 ). The purport of the divine rebuke to Cain was this, "Why art thou angry, as if unjustly treated?
If thou doest well (that is, wert innocent and sinless) a thank offering would have been accepted as a token of thy dependence as a creature.
But as thou doest not well (that is, art a sinner), a sin offering is necessary, by bringing which thou wouldest have met with acceptance and retained the honors of thy birthright." This language implies that previous instructions had been given as to the mode of worship; Abel offered through faith ( Hbr 11:4 ).
      unto thee shall be his desire--The high distinction conferred by priority of birth is described ( Gen 27:29 ); and it was Cain's conviction, that this honor had been withdrawn from him, by the rejection of his sacrifice, and conferred on his younger brother--hence the secret flame of jealousy, which kindled into a settled hatred and fell revenge.

Jamieson, Fausett & Brown

[malakhi]

but can you show me where it says in the scripture that an unsaved man can do anything to please God?

But if a wicked man turns away from all his sins which he has committed and keeps all my statutes and does what is lawful and right, he shall surely live; he shall not die.
None of the transgressions which he has committed shall be remembered against him; for the righteousness which he has done he shall live.
Have I any pleasure in the death of the wicked, says the Lord GOD, and not rather that he should turn from his way and live? (Ezekiel 18:21-23)

The repentence of the sinner pleases God.