Hacker News new | past | comments | ask | show | jobs | submit
BLKNSLVR | on: Microsoft Outlook/365 still down after 12 hours
How many Microsoft employees are working on this? It's concerning that one of the biggest, most valuable companies on the planet, that runs services internationally, can have one of its primary services have gone down for this long.

6 hours ago they seemed to not exactly understand what the problem was, and that was 6 hours into the outage.

This is concerning.

sobiolite | parent | next
Not so long ago, I was assigned to a project that had to be built on Azure due to client requirements. Because we were underhanded and it was an MVP, we'd elected to use PaaS as much as possible: Postgres Flexible Server, Static Web Apps, Easy Auth, etc. On paper, these all looked fine. They had lots of documentation, example projects, GitHub repos, blogs, roadmaps, etc. Then we started building and running into issues, and we realised a bunch of these offerings were basically unmaintained. There were features that had been stuck in "preview" status for literally years. The blogs and roadmaps were no longer updated, and issues were piling up in GitHub.

I started digging into the historical blog posts, GitHub comments, etc. and I realised everybody at MS who'd been working on them, SWEs, PMs, architects, etc. had either left the company, or moved to other teams. And the one constant about the internal moves was that people were now working in some "AI" related project or product. Literally, I didn't find a single person who wasn't now working on AI.

Clearly, there has been a huge internal push to get into AI. Resources have been reallocated, and anything non-AI related has been starved. And since working on an AI project is probably considered the only route to advancing your career, anybody with any ambition has moved to a team doing something AI-related. The result is that huge numbers of products and teams doing vital but non-AI related work have been hollowed out, and now we're seeing the consequences.

loading story #50483559
loading story #50483178
pjmlp | parent | next
Let me put this way, the Visual C++ standard library has officially one FTE working on it, everything else are open source contributions.

WinUI 3.0, WinAppSDK, and related CsWinRT, C++/WinRT are in a ridiculous position six years after Project Reunion announcement, due to ever meagre resources.

XBox app keeps under delivering user experience, and bugs on patch Tuesdays are getting a joke by now.

All this in a four trillion valued company.

ctmlr | root | parent | next
Maybe acting like jerk-offs got them to four trillion but there are consequences to burning all your social capital. A large camp of software developers have genuinely hostile feelings towards Microsoft, I would bet that this affects the quality of talent they can hire and these are downstream affects.

Lately it seems like more non-techies are souring on the company.

loading story #50480582
loading story #50480062
loading story #50481378
fifticon | root | parent | next
Tellingly, I will not go to MS for UI stuff. When I have to anyway, I use their decades-old APIs. It is a canary with regards to microsoft, that they have no known champion for their UI stuff, no trusted face for their UI stuff; the company all just appears to coast along on "I assume someone is handling it?" The main microsoft employee I trust is Raymond Chen, they have no current generations of Raymond Chen / that we know of.
loading story #50480424
loading story #50480498
Cthulhu_ | root | parent | next
This is what I don't get about big companies and their software; you'd think every application, every moving part would have a (small?) team of core maintainers, people that get paid well to know everything there is to know about something, ranging from (in microsoft's case) Paint to a particular configuration panel.

I get that this doesn't scale though - a lot of features are just "done" at some point and keeping one or more people whose sole job is to maintain this may lead to them bolting on features just to stay relevant / stop being bored. But big tech can easily afford to keep people like that on.

(I mean they keep (kept?) tons of middle management on, who knows what they even do? Not emailing right now that's for sure am I right?)

loading story #50481441
loading story #50480951
logicchains | root | parent
>the Visual C++ standard library has officially one FTE working on it

One guy but it's Stephan T. Lavavej the guy who the "STL" was literally named after.

EmX_3000 | parent | next
Mythical man month, maybe throwing everybody in the company at it isn't the right thing to do. But they must be in a panic, and their own communications internally may well be hampered, could be MS everything is out for MS as well as everybody else.
TheOtherHobbes | root | parent | next
Someone's going to have to write the Mythical AI Month after this.

If you lose 90% of developers to AI, but everything breaks, how much money have you saved?

The underlying moral issue is that the management and money people have contempt for people who don't manage and don't have money. Which includes customers and engineers.

It's not just an inhumane system - although it is, that, brutally - but it's also technologically dysfunctional and ultimately unstable.

Take8435 | root | parent
"This is on the money!"

Sorry, had to.

flir | root | parent
I've seen the response to an emergency be "get every dev in a big room with some non-technical middle managers to direct them, and have everybody talk at once". My jaw was on the floor.

"When in danger, when in doubt, run in circles, scream and shout".

nikanj | parent | next
In our field, once you understand what the problem is, the problem is usually solved. Unless it's broken hardware, obviously
justinclift | root | parent
But Copilot is down too, so maybe they're screwed? :)
InsideOutSanta | parent | next
The ones who actually knew how the system works were too expensive to retain, so now it's just some interns typing frantically into Claude Code.
digikata | root | parent | next
"Too expensive" as measured by bean counters that don't understand the actual risk profile - or do understand it but have such a short window of strategic concern it leads to this. It would be an interesting analysis to stack up costs to Msoft vs cost of labor cut as this outage evolves.
StingyJelly | root | parent | next
Company policy is to use copilot but copilot is down so they are just hitting F5 and hoping
aucisson_masque | root | parent | next
Haha this is ridiculous and yet there is some truth in it.
kakacik | root | parent
It may not be true now but 10 years down the line? This is disaster in the making yet while most people recognize that (when filtering out most fanatical pro-ai folks claiming everywhere how they get 10x or 100x work done) nobody is able to do anything. Sort of legacy technical debt being delivered but of different kind than usual, that train that will hit you since you stand on the rail line yet its moving 5kmh and you see it from 200m, but unable or unwilling to move away (Austin powers scene with steamroller does it better than me [1]).

Imagine if say BMW decided they don't want to manufacture their motors, suspensions, drivetrain etc. Outsourcing 100% of that knowledge to external suppliers, for all models, from now on forever. How do you think market would reacted to such move ie by valuation of stocks?

[1] https://www.youtube.com/watch?v=y_PrZ-J7D3k

loading story #50481702
loading story #50481064
SillyUsername | root | parent
This should be the top comment, my eyes are watering to this one
InsideOutSanta | root | parent
Oh shit: https://status.claude.com/incidents/nn11wk485zfn

Microsoft can't fix Outlook because they don't have Claude, and Anthropic's employees can't fix Claude because they can no longer log in without Outlook?

briansm | root | parent
Ah, 'The Gift of the Magi', a classic Christmas tale.
micw | parent | next
> How many Microsoft employees are working on this?

all 3!

digikata | root | parent
They're more effective than ever because with this outage, every manager above them can't email to ask for status every 5 min
pferde | root | parent | next
Alas, Teams are working, or at least the chat and voice call portions of it. So managers are in a calling and chatting frenzy right now, pulling every body else in.
loading story #50480736
motionlessveloc | root | parent
Fortunately they can still ask on Teams!
loading story #50482915
loading story #50483075
karlkloss | parent | next
How may Microsoft employees are left anyway?
phoronixrly | root | parent | next
The lights are on but there's no one home
theflyingelvis | root | parent | next
Your mailbox is not your own
rdtsc | root | parent
The wheel is still spinning but the hamster is dead.
wolfi1 | root | parent
I guess the old light bulb jokes about MS will not be applying anymore
boudin | parent | next
/s Their internal AI is full steam on it. The issue is just that their internal AI infrastructure is down too, so AI just need to fix itself first. Then it will focus on fixing the issue. Oh, wait...
wooger | parent | next
And which made a load of staff redundant in the last month.
surgical_fire | parent | next
This is HN. We should be asking why frontier models couldn't still dolve this! How many parallel agents does MS have working on this?
| parent | next
{"deleted":true,"id":50479677,"parent":50479629,"time":1794555296,"type":"comment"}
DarkCrusader2 | parent | next
I don't understand why they haven't asked copilot to fix it yet, make no mistakes.
mint5 | root | parent | next
Don’t be silly, they tried that but copilot is/was down!

That’s why it’s taken over 12 hours now, no one’s left that knows without copilot’s help.

xyst | parent | next
the vibe coding group in charge of this product has decided to delete the production database and backups
krige | parent
Maybe Nadella is still trying to figure out which of the 8000+ sacked employees knew how the system works /s
sterlind | root | parent
he was such a nice and effective CEO the first few years. he's really changed.
kakacik | root | parent | next
Was he? I struggle to see it even in hindsight, and definitely his moves were bunch of FU to common users. Maybe in some specific area you care about, but not their core products like windows and office
cyberax | root | parent | next
Not really. He killed the QA department immediately and consciously put MS onto the "slide into mediocrity" path.
motionlessveloc | root | parent
Google killed its test engineers (or, rather, forcibly merged them into SWE) at around the same time without measurably denting uptime.
pjmlp | root | parent | next
Even with some of the Microsoft <3 Open Source, I do miss the Steve Balmer era in regards to Windows development experience.
bell-cot | root | parent | next
Executive compensation corrupts, chief executive compensation corrupts absolutely.
Markoff | root | parent
yeah very effective, he fired 90% of our Windows Store team (reviewing the apps) in the first year to replace people with automation, I survived up to the last round of firing after which they kept ~20 employees out of original ~200