The heart with writing on it symbolizes the Prophecy of the New Covenant. The key passage comes from Jeremiah 31:31-34:
"The time is coming," declares the LORD, "when I will make a new covenant with the house of Isreal and with the house of Judah.
It will not be like the covenant I made with their forefathers when I took them by the hand to lead them out of Egypt, because they broke my covenant, though I was a husband to them," declares the LORD.
"This is the covenant I will make with the house of Isreal after that time," declares the LORD. "I will put my law in their minds and write it on their hearts. I will be their God, and they will be my people.
No longer will a man teach his neighbor, or a man his brother, saying, 'Know the LORD,' because they will all know me, from the least of them to the greatest."
Another verse that speaks to this is from Ezekiel 36:26-27
"I will give you a new heart and put a new spirit in you; I will remove from you your heart of stone and give you a heart of flesh. And I will put my Spirit in you and move you to follow my decrees and be careful to keep my laws."
What an incredible gift of grace that God has given us: a new heart and His Word written on our hearts! Thank you, God, for not giving up on your people and for this New Covenant fulfilled in Christ.