Unknown idle issues

Hello,
I purchased a 1982 yamaha xj750 seca and it used quite a bit. It has a really high idle at 3-4k rpms. I originally had no idea what I was doing as this is my first bike. I cleaned the carbs completely and it hardly fixed it and I started fiddling with the syncing screws and that helped a bit, but I changed the oil and the idle shot back up to 4k rpms and it was sluggish.
I did a lot more research and got a carb rebuild kit, a syncing kit and new intake boots and gaskets. I rebuilt the carbs and replaced the intake boots and gaskets. That really helped, however while trying to sync the carbs, the idle would jump between 1-4k rpms. I expected the jump, however when I was able to get it to idle around 1k rpms, I would shut it off and take a break, but when I tried to start it up again it would have a really hard time and then the idle would jump back up to around 2,500 rpms.
I haven't touched the bike I'm a while as it became cold. I just wanted to know if I'm on the right track or not. I also interested in any advice as I haven't found anyone with this problem.