Your problem might be that when you leave the car for a couple of hours the fuel inside the float bowl evaporates. That's why you need to spray it with carb cleaner so the engine can run and the fuel pump can "pump" fuel back into the float bowls. That can also be the reason why when the engine is warm you can stop/start it fine without problems, cause the fuel hasn't enough time to evaporate. Next time it happens try and look into the sight glass to see if there's fuel in the bowls. Just my 2 cents thou, just a backyard mechanic, feel free to correct me if I'm wrong.