Going to sleep

For the event mission, I slept for a while after a long time, and from noon, I was playing with gem boosters, but none of that was counted. When I inquired, they basically said they applied a penalty because there was too much idle time. I don't really know what to say when they tell me I wasn't operating it, given it's sleep tracking.