If you don't do the sphere in a certain amount of time, will Valygar complain or even leave? If yes, how long do I have to complete this quest?

I'd rather wait until Chapter 6 (when my party would be complete), but if I can't do it with Imoen, I'll give it a try without her.

TIA.

What about travelling with him to the sphere, opening the door and then removing him from your party and picking him up in chapter 6?

I'm fairly sure he will complain eventually. It's possible that proceeding on the path to chapter 6 will put a stop to his complaining, in the same way that romances are halted.

It apparently isn’t timed.

He never complains.

Though if you are doing a no reload game you might want to go and open up the sphere with him…you don’t’ have to go into it or anything