I don't recall there being any formal criteria. There was nearly always a good collection of game compatibility bugs assigned to me, which I guess wasn't surprising since that was one of my jobs after all. Sometimes it was easy to figure out; sometimes it was hard. Sometimes it took a day; sometimes it took several days.
The bugs were assigned priorities and severities by the person who found the bug. For example, something that prevents the game from running is more severe than a visual glitch in a game that otherwise seems to run fine.
If the game was a major title, it was more important to fix, compared to a game that didn't sell as many copies. For example, DOOM was a pretty high priority to get working.
But overall, I was allowed to exercise my own judgment as to how much time to spend trying to fix each game. The fact that we had MS-DOS Mode available meant that there was a fallback if I couldn't figure out what was wrong, or if I figured out what was wrong and there was no practical way around it.
I"ll take the next few Mondays discussing some of the things that I had to deal with. It's cheaper than therapy.