Granite is mostly quartz. Quartz has a Mohs hardness of 7. The majority of the rest of granite is feldspar, which has a Mohs hardness of 6, so it's softer. Quartz's Knoop hardness is about 820.

Most quality kitchen knives are made of 440C stainless, which is a very hard steel. It has a Rockwell hardness of about C57 when hardened and tempered, which is more-or-less equivalent to a Knoop hardness of 670.

I don't know what's used to polish granite, nor do I know if the other minerals in yours particularly are soft. But granite is, by far, harder than steel.