If it has to be your drill press, you might consider an inline planetary gear reducer:
I have this for a job. It is used by ice fishermen to slow down their battery drills to use with large ice augers. Your 540 rpms would be reduced to 150 and the torque increased by 3.6 times.
Someone would need to modify the output shaft to take an annular cutter, hole saw or whatever. It would need a simple torque arm too. I would modify it in a way to make it as short as possible. You may still need a guide bushing clamped to the part being drilled to steady things.
Look for Clam power drive. You can get them for about 70 bucks.
It used to be common for use to do "embiggening" jobs, where a part had an undersized hole that needed to be larger. It was so cheap and easy for us because I had a program for every common hole size and the plasma cutter had a taper and floating head. Just crash the plasma roughly into the hole, wiggle the plate until it dropped in, zero and start the cut.
Go to your local welding supply and rent a mag drill. You may have to buy the correct rotabroach bit but it will work enormously better than a hole saw. 1-1/4" is right in a mag drill's sweet spot. Here's a closeup of my little Steelmax (Poland) cutting a 2-1/16" hole right through a weld seam:
That is a curved surface, so I have a little "table" I tack onto the tank so the mag drill can attach to a flat surface.
Just to throw another option out there... If this is on new stock, perhaps consider sending it out to either a local shop with a laser or plasma table or one of the newer prototype places like oshcut.com or sendcutsend.com on the web. It might be cheaper and easier to do that as your drill press seems a bit high on rpms to run a hole saw like you need. I have rented a mag drill on occasion too, but you need to supply the bits for the most part. Might be cheaper to send it out to be cut.