Our visit was influenced by very high temperature, my watching vlogs that showed people raving about the food they were overeating (before the trip was arranged), & my surprising lack of skill in dealing w/ a new country for us.
The sites: How much will one enjoy seeing them? I think it depends greatly on how much that is similar has been seen by the tourist. Like tourists in Europe who loved seeing cathedrals that are not the greatest at the beginning of their trip & hardly enjoyed the greatest cathedrals at the end of their tour, I at 72 am difficult to amaze. I am pleased moderately by almost everything. Trying to be useful as well as honest, I'd say when the temperature is not around 37 C, the highlights of Penang should be impressive enough for most of us,but are not something one makes a trip for on their own.
Nit is blind & after taking her to many temples, I am amazed that she picks up a vibe at special places usually; & I am amused that after I tell her whether the statue of Buddha is standing, lying down, etc., Nit's prayers are not the same for each position. If your companions don't keep you so occupied, I'd say the tourists at Penang are not bothersome (except the amount of them on Friday - Sunday) & sometimes fun to watch.
Things to buy: Not the place for that. I did not search hard. I hoped for some good traditionally handwoven silk. Did not see any.
Getting around: Buses were disappointing. Bolt (Asian Uber style) was excellent. We mostly walked & were challenged by lack of signs directing one to sites, even no signs at some sites, & by people not knowing where they were & what was around them. People working at hotels did not even know the address of the hotel where they worked.
Eating: Known as a food paradise. I'd say, it is not really. Lack of vegetables w/ almost any meal. We found good food not terribly more expensive than Thailand, but after the hyperbole of blogs, it was difficult not to be disappointed. Really refreshing herbal teas, freshly prepared carrot juice (& others) mixed w/ some water & simple syrup (sugar & water) were a pleasure (& cheap). Some pastry was very good (not cheap but worth it). Some bakeries & specialties recommended by vlogs were ordinary (e.g. Portuguese cheese/egg tarts). The "best" ($$$$) durian was overwhelmingly strong for us & milder durian was not better for us than what we eat in Thailand. We did not take the trip to durian farms which looks like a nice outing.
The very best: For us it was time w/ OCTO. A kind, generous, charming host, who treated us w/ his company & a good breakfast of Indian food that was not only quite tasty but included an Indian bread that I had not had before (& I thought I was an accomplished glutton

Overall: I'd say if not going to beaches, Penang is a place for 2 or 3 days. I wanted to be in the Heritage section, but a beach hotel or one w/ a good pool, makes more sense & for a couple of more days, if affordable. Of course, various factors effect our decisions (e.g. it was time for me to make a visa run & to be at a new place). Cheers