Mast step wear: is this a problem?

I finally got around to looking inside the mast step of my recently-acquired 1973 Minifish, and noticed that the inside of the mast step is gouged or worn away on both sides near the top. Is this something I should be worried about, and try to fix somehow? I've never done anything with fiberglass, and have no real desire to start now. Most of the inner surface is smooth, like the outer hull, but in these two places it is bumpy. Here are some pictures:
[ATTACH type="full"...

Mast step wear: is this a problem?