Background: A Routine Test That Wasn't
I'm the office administrator for a 300-person software company. Since 2020, I've managed all facilities and IT procurement—roughly $250,000 annually across 12 vendors. I report to both operations and finance. In early 2024, we consolidated our data closet infrastructure and standardized on Schneider Electric UPS systems. We purchased two Schneider Electric Easy UPS 3S units for our primary and backup data closets. The plan was simple: install them, test the failover with our house power transfer switch, and then get back to work.
If you've ever managed a data center, you know that sinking feeling when a plan falls apart. Ours fell apart in March 2024.
The Test: Why We Were There
We scheduled a full power transfer test for a Tuesday afternoon. The goal was to simulate a mains failure and confirm that the UPS would carry the load while our portable generator kicked in. We had rented a 30 kW generator and had it wired to the building's transfer switch. The facilities team was on standby. I had a checklist: verify load, notify stakeholders, start generator, kill mains, watch UPS, restore mains. Standard stuff.
We started at 2 PM—actually 2:30 because of a meeting overrun. The first sign of trouble was when I tried to log into the UPS web interface to confirm battery health. The login prompt appeared, but my usual credentials didn't work. I tried the default admin/admin, admin/password, and a few others. Nothing. I pulled up the manual (which I should have read earlier) and found that the Schneider Electric default password UPS for this model was a generic string printed on a sticker inside the front panel. We had never changed it. That was mistake number one.
The Discovery: A Default Password Is a Red Flag
It's tempting to think that a default password is harmless if the UPS is on an isolated network. But our data closet shares a VLAN with the building management system. Leaving a default credential is like leaving your front door unlocked because you live in a "safe neighborhood." I called Schneider support, and the technician walked me through resetting the password—but it required a reboot of the UPS management card. We couldn't reboot during the test. So we proceeded with the test, planning to change the password immediately after.
"Default passwords are there for initial setup only," the tech told me. "They're a huge security risk if left unchanged. We recommend changing them before the unit is even racked."
That was the first lesson: assumption failure. I had assumed the vendor set a unique password during installation. They didn't. We never verified.
The Turning Point: When the Transfer Switch Didn't Switch Back
We killed the mains at 2:45 PM. The UPS picked up the load seamlessly—no flicker. The generator started, and the transfer switch moved to generator power. So far, so good. Then we restored mains. The transfer switch should have moved back to utility power and signaled the UPS to recharge. But the switch stayed on generator. The UPS remained on battery, draining. We waited 30 seconds. Nothing. The facilities manager started panicking. If the generator ran out of fuel, we'd have 10 minutes of battery left before a hard shutdown.
Someone on the team suggested grabbing a Cen-Tech battery charger from the maintenance closet to "boost" the UPS batteries. I shut that down immediately—that's a sure way to void the warranty and possibly start a fire. But we needed a solution fast.
I went back and forth between two options: cancel the test and manually bypass the UPS (risking downtime), or keep the generator running and troubleshoot the transfer switch. The upside of pushing through was learning what went wrong; the risk was a full outage during business hours. I chose to push through because we had vendor support on the phone. That was the binary struggle.
Turns out, the transfer switch had a faulty relay. The UPS was fine—it was doing exactly what it was supposed to do: stay on battery until it received a stable utility signal. The real problem was the switch. We manually overrode the relay, restored utility power, and the UPS recharged. Total elapsed time: 3 hours. We narrowly avoided a shutdown.
The Resolution: What It Cost Us
The final bill: $8,000 for emergency after-hours support from an electrician, $2,500 for the generator rental (extended by two days), and $1,500 in overtime for the facilities team. Total: $12,000. The root cause was a combination of two preventable issues: a default password we never changed, and a transfer switch we never tested end-to-end before the live drill.
We also learned a painful lesson about being penny wise and pound foolish. Earlier that year, I had skipped the $200 network management card for the UPS because it seemed unnecessary. Without it, we had no remote alerting—we only discovered the battery drain when we physically checked the LCD. That $200 would have saved us at least an hour of panic.
And yes, we did have to quickly find where to buy portable generator rentals in a hurry. We ended up using a local equipment rental company, but not before I wasted 20 minutes searching online. Now I keep a list of pre-vetted vendors for emergency power.
Lessons Learned: Prevention Beats Rework
After the incident, I created a 12-point checklist for every UPS deployment. It's saved us an estimated $8,000 in potential rework over the past year. Here are the key items:
- Verify and change all default passwords before the unit goes live. Document the new credentials in our password manager.
- Test the full failover chain—UPS, transfer switch, generator—at least twice before any live drill.
- Read the manual for any model-specific quirks (like the default password location).
- Include remote monitoring in the initial purchase, even if it seems optional.
- Never use automotive battery chargers on UPS batteries. Call the vendor.
5 minutes of verification beats 5 days of correction. I learned that the hard way. The schneider-ups hardware is reliable—but only if you configure it correctly.
To be fair, the vendor support was excellent once we got through. The problem was us. We assumed “install and forget” would work. It doesn't. In power protection, prevention is the only cure.
