Submitted by “Have you ever wondered what really happens to those Windows error reports you can send to Microsoft whenever a Windows app crashes? How many reports it must receive before taking action? Or whether it’s worth your time and effort to send duplicate reports if the error occurs repeatedly? I did, and I asked Microsoft. Unfortunately, after a week and a handful of assurances that they were working on responses, the software giant refused to speak with me.”
Microsoft doesn’t have time to patch some security vulnerabilities for years, specify patent infringements they allege, comply with the EU trade union, or fulfill all their promises for Vista (ie WinFS, etc). I love OSAlert, and it’s content, but seriously… this is more of the kind of news that ‘us’ (OSAlert readers) are already completely desensitized to. OSAlert (and many other reputable OS info sites) posts stories like these about MS on a daily basis. Just one more way MS proves that everyone they serve is simply another number, just a supplement to their bottom line.
Edited 2007-07-10 18:50
“this is more of the kind of news that ‘us’ (OSAlert readers) are already completely desensitized to.”
Should be ‘we’ (OSAlert readers)…
I hear many people tell me they “never” submit data to Microsoft. I always do it because I want them to get debug info to make Windows better.
I’ve sent my last dump though. I never really thought about it, but if Microsoft won’t share this kind of harmless info, then why should I share with Microsoft what’s happening on my computer, which is clearly more invasive than these questions that are completely legitimate.
This just reaffirms what I’ve thought for some time, that Microsoft simply cannot be trusted to ever make the right decision for the user, and can be counted on to always make the right decision for their own selfish needs. I wonder what will happen if we pose the same questions to Apple?
Quoting Adam Scheinberg:
“I wonder what will happen if we pose the same questions to Apple”
This doesn’t answer some questions but gives you the anatomy of a crash log/report in Mac OS X
http://developer.apple.com/technotes/tn2004/tn2123.html
I don’t expect big companies like Microsoft and Apple to answer each individual question but I understand what the author means.
There is a video on Channel9 answering each and every one of these questions, as well as the future of error reporting in Windows.
http://channel9.msdn.com/ShowPost.aspx?PostID=118625#118625
Edited 2007-07-10 20:13
I remember that video, but I can’t find it now…do you know where it is offhand?
EDIT: Thanks for adding the link.
Edited 2007-07-10 20:17
with those bug reports…
*nothing*
that’s why they won’t give you an answer. can you imagine how embarassing that would be to say it in public?
if 100 corp. customers report 10 bugs (through special channels), they’ll make more $$ on fixing those 10 bugs than they would if 10000 home users reported the same 10 bugs via the automatic crash reporter.
Your source please if you may?
“if 100 corp. customers report 10 bugs (through special channels), they’ll make more $$ on fixing those 10 bugs than they would if 10000 home users reported the same 10 bugs via the automatic crash reporter.”
Huh? A bug is a bug is a bug, and an hour spent fixing your so called “corporate” bug costs the same as fixing a “home user” bug. And I’m extremely curious to hear your reasoning about how MS makes money from bug fixes considering all of their patches/service packs/rollups are free.
Regardless of what channel a bug gets reported through, MS doesn’t make any money off of bug fixes. In fact, the dynamics are such at MS that the product support groups actually bill the product teams for bugs they get calls on until the bug is fixed, so the product teams actually lose money on bugs, hence the incentive to get them fixed quicker.
i don’t know how you figure that. imagine this…
corp customer finds a bug during testing. stops rollout due to bug. MS fixes bug, corp customer can then continue to roll it out on more machines, and buy more licenses (1000 more copies of Vista sold to finish the rollout at SmallAndBigBusinessRUS). i’d guess this is a pretty common scenario…
compared to user who finds bug, gets bug fixed by MS at the cost of 10 man hours lets say. user *MIGHT* (if planets are aligned properly) buy 1 or 2 more copies because of this.
sounds like pretty good reasoning to me.
besides, my original post was pretty sarcastic despite there being some possible truth to it. i never claimed it as fact. it wasn’t my intention to get anyone’s fact-panties into a wad. obviously they do SOMETHING with those reports. c’mon.
There was a time that I installed some audio software on my computer. Later, I found that every time I ran a particular app and executed a particular operation, the app crashed. (I didn’t relate it to the audio software that I had installed days earlier.) After multiple crashes, I finally sent in the error report. Much to my surprise, the error-report tool immediately responded with information as to the cause of the problem and how to fix it. (The audio software, which the error-report tool actually explicitly identified, had altered sound driver settings or whatever (I forget the details).) I tried the suggested fix and it worked. (I later uninstalled the problematic audio software altogether. :p)
So if your error report matches an entry in their database, you’ll get immediate info as to the cause and fix. If it doens’t match an entry in the database, the data you send will lead to creation of such an entry if practicable (i.e. if enough people send reports about the same problem and the cause can be identified).
Edited 2007-07-11 00:00
I’ve also heard of problems being automatigically fixed as a result of sending an error report. A tech support customer had a problem a few months back with his PC bluescreening – about 5 minutes before I was due to head over to look at it, I got a call from him saying:
“It crashed again and this time I sent an error report. Then I got a message saying a patch is available for this problem and asking if I would like to download and install it. I hit yes, and it’s been working fine ever since.”
It was a nice surprise, I guess I’ve just been dealing with MS software for long and have come to expect their error messages to be completely useless (OE wins for the error message: “Unable to import Internet account settings file because: a error occurred”).
Of course, this is second-hand based on a phone account from a less-technical person, so grains of salt and all that.
While I don’t expect them to have an answer it would have been interesting if they had responded with some basics like how many crash reports they roughly receive in a given time frame.
The title of the article is misleading. They did speak to Al Sacco several times. What he is forgetting is that Microsoft is not at liberty to disclose how the information is being used but if the ultimate end goal makes Windows better than I’m all for it. It was more of a bunch of “curiosity killed the cat” questions that Microsoft doesn’t have time for. I wonder which email address he tried to contact Microsoft at, and if he even got the right team or just the general PR people. He’d be better off asking the developers if they use the information and how they use it.
Nonsense. He doesn’t want specifics…. he just wants to know if he should resubmit bugs and how many reports it takes for action. That is not sensitive information any way you cut it unless they are discarding the information outright. It sounds to me like they simply aren’t using it, which is why they are hiding behind a mask of “no comment.”
And he received the answer – yes, the more a bug is submitted, the more attention it will receive. To ask a goofy question like “And how many does it take before it is acted on?” is ridiculous. Did he really want to hear “Well, when we receive 137 reports we act.”?? Obviously there is no cutoff, just a general rule (repeated here again) – the more a bug is submitted, the more attention it will receive.
A non-article, IMHO
(Yes, he should resubmit, at worst the system will take care of it automagically.) I would suspect the reason for not releasing the numbers is that they vary per product/team, so it’s not something that could be measured. Also bugs are of many different kinds. Debugging a crash that happens in a race condition could take quite a few reports to be helpful for the developer. On the other hand there are other bugs which could be fixed in 5 minutes (except for the rigorous testing process of course) with just one crash report in hand.
(Also being a developer forced to respond to support requests sometimes I could very well understand the pain for the MS people to answer such questions. You have to keep in mind that the people working with the WER system are developers and they consider other metrics important than a basic user. I often find myself unwanting to explain the technical details to “basic users” because the technical barrier between my thinking and theirs is relatively high and it would take a lot of time for both of us.)
“It sounds to me like they simply aren’t using it, which is why they are hiding behind a mask of “no comment.”
Adam, I’m surprised by this statement. Why would MS go through all the trouble of allocating hardware resources (machines, bandwidth, etc), code to run it, etc for something as complex as this, and then simply not use it?
Anything that software developers can use to make their products more robust is a godsend, and I know for a fact from my many MS developer friends that WER is one of the ‘heart and soul’ apps that MS uses to make bugfixes. They get an enormous amount of data on a daily basis, and they also have a very complex routing system in place so that reports go to the correct teams/get triaged/etc.
Another reason they are probably reluctant to give out specifics is that then gives all the naysayers another moving target to hone in on, i.e. “well, MS said they get 10,000,000 bug reports a month, so why did they only release 5 patches last Tuesday?”
Just to clarify, these are not necessarily Microsoft bugs, but any app that runs on Windows, including Firefox, Abiword, Gimp, etc. I think Ubuntu is trying to accumulate the same kind of info also in a future release (IIRC).
[EDIT: spelling]
Edited 2007-07-10 20:43
“Just to clarify, these are not necessarily Microsoft bugs, but any app that runs on Windows, including Firefox, Abiword, Gimp, etc.”
That is correct, and any company who has a Verisign certificate can view crash data collected by Microsoft on their products for free online. Yes, free. Hell, half the time MS will even help to fix it. Why on earth would they do that? To provide the best possible end user experience on Windows for their customers.
Edited 2007-07-10 20:42
The smell of the best user experience is in the air.
Frankly I think that both MS and Apple mostly use this as a warning system – if they start to get way too many similar reports (and, like, millions of similar reports) then they know something’s loose. They probably don’t pay attention to the background noise, applications do crash from time to time for different OS-unrelated reasons (hardware failure for example).
I am wondering how many potato chips are being consumed by Microsoft employees daily, weekly and monthly. Maybe I should write them and email and ask. If they won’t reply at least I’ll be able to write an article about how evil Microsoft mistreated me and post it on OSAlert.
The Microsoft page actually states some numbers:
“Microsoft has benefited from Windows Error Reporting during the development and after product release. The Office product group fixed 50% of all reported errors in Service Pack 2. The Visual Studio product group fixed 74% of reported failures by Beta 1 for Visual Studio 2005. And the Windows product group fixed 29% of problems reported on Windows XP in Windows XP Service Pack 1.”
I’m not surprised they don’t release the number of reports received because the reports also cover third party software from other vendors and even after filtering them it’s not good for their marketing.
Some interesting read:
http://blogs.msdn.com/larryosterman/archive/2004/06/01/145627.aspx
Here’s another link to compliment what you have posted. This one is mostly related to overclocking and how it causes error reports:
http://blogs.msdn.com/oldnewthing/archive/2005/04/12/407562.aspx
For those of you that think they do nothing with the data: think again.
1. http://blogs.msdn.com/mikekelly/archive/2005/02/03/366861.aspx
2. http://blogs.msdn.com/oldnewthing/archive/2005/08/10/449865.aspx
3. http://blogs.msdn.com/oldnewthing/archive/2005/04/12/407562.aspx
4. http://blogs.msdn.com/chris_pratley/archive/2004/02/04/67276.aspx
5. http://blogs.msdn.com/larryosterman/archive/2004/06/01/145627.aspx
Want more links?
Edited 2007-07-10 20:31
… to “Tech Writer Incapable of Using Google to Find Freely Available Information”.
Yuck, I hate trying to read articles when sites split them over several pages and only including a few paragraphs per page.
This is the trend, posting controversial/microsoft hate topics and splitting it over far too many pages to increase the exposure to ads. Oh yeah, and being unable to spend 5 minutes searching for the answers to questions that OSAlert readers were able to find!
sheesh
Though this article has its merits by the nature of the subject matter since most Windows have been there, I find it to be malicious.
If he wanted to find out how the process works they are enough sources on the web.Secondly, the manner he asked the questions anyone could clearly see there was a buildup to potential scandal.
Next time he should ask questions on his own behalf , for example , what happens to MY report and what would be a reasonable response time for IT? Then readers could have drawn their own conclusions.
“I like to think that my post could’ve led some of you to submit more Windows error reports. Or at least understand what’s happening when you do. Instead, I’m left with a bad taste in my mouth–and unanswered questions.”
Astroturfing ?
Frankly I think he deserved what he got.
Edited 2007-07-10 21:26
Being responsible for processing customer error reports in our tiny software company, I think I can sched some light on topic. Although Micorosoft has many orders of magnitude more customers and errors to recieve, I think general ideas are same (of course MS needs to use automatic preprocessing of reports, which is relatively simple task).
1. Reports overall count
In our case reports count is not stable and depends primarily on new releases schedule. I’m sure that this is same in every software comapny.
We release our software major versions about 3-4 times a year, there are usually some weeks after release when we’ll receive many reports. These are mostly about bugs, slipped through our internal testing; each bug causes lot of similar reports, easily categorisable visually (and probably automatically too, if we ever need such automatisation). We release bugfixes ASAP, sometimes every day (after major release).
I can’t say exactly, how many error reports we get from our about 1000 users. Some day none, some day five, rarely more too. Fortunately errors (bugs) count decreases over time (especially after we reorganized our internal testing procedures some years ago
)
2. Reports importance
What more reports we got about the same bug, then important is to fix that bug. This is natural IMO:)
Well, there are of course different bugs; crashes are most important, invalid tabstops or focus placements are least important.
Another category are single incident reports – I mean reports about bugs, which happen only once for single customer. Usually these are caused by errors in not our software – network errors, antivirus false blockings, hardware/driver errors and similar.
Usually we ignore those reports – but put them in waiting list; if we encounter similar error pattern another time then such error goes into investigation.
And there are lot of common errors, caused by another products too, but easily fixable – like some printer drivers errors or some specific OS problems. Such errors with solutions will go into FAQ (which noone seems to read – almost every day I need to point to our FAQ in response to some error report).
3. Privacy
Poses no problem. Our error reports are in plain text, customer may opt to not send these, if he/she finds something private in report content. Not happened so far
—
About Microsoft – their error reporting service has surprised me sometimes. I’ve seen case, when after submitting error report to Microsoft they offered correct solution on their response page – namely to update video drivers. Error itself wasn’t so usual to qualify as driver issue at all – but MS could decode sent information and give solution. Since this incident I’m a bit eager to submit error reports to MS
…on their wire, cause a failure, send the report and see if they can translate the packet data (assuming they are encrypted). Then you would know what is being sent.
As far as what does MS do with it… I am SURE they do some sort of statistical reporting to flag the more critical problems, just as the author was told. The other questions he asked are none of his business, really.
If he wants immediate action he can join MSDN and report bugs that way, can’t he?
You don’t need to go to this extent. WER collects crash dumps in c:windowsminidump (Vista). You can use ntsd or WinDBG to debug the issue. It will be a bit hard to do without the private OS symbols, but you can get a lot of information even without full symbols (I, for instance, was able to reverse-engineer minesweeper using these tools and I’m not that great at it). You can certainly see what information is being sent there. It could be somewhat sensitive, because it contains significant pieces of the state of the dying application (including what data you put into its memory).
Debugging a minidump is not easy stuff…. sometimes they get mysterious dumps which require more information (you’ll occasionally see a box asking if you want to send even more information to Microsoft). In that case, they probably get a full dump with more system information.
What happened here is probably that the bug is in Adobe’s software (which is fairly complex). The IE team likely doesn’t have extra time to deal with external code and are loath to say anything bad about a partner’s software no matter how crappy it is. (After all, it’s the legacy crappy code that continues to work far longer than it should that keeps the Windows platform so popular).
Good points.
That is not the orthodox way to get customers into the “software as a service” thing.
http://www.pcworld.com/article/id,123364-page,1/article.html
While in XP you were hard pushed to know that your reports were makin any difference, the same can not be said with Vista.
The Problem Reports and Solutions centre in Vista is excellent.
You get updates with issues you have found, it even links you to updated drivers of websites to get patches for failed products with reasons for the failure.
As to this article; it honestly seems like a non-entity to me. Why should Microsoft tell anyone anything about this process? They say it won’t be used to personally identify, it won’t be sold on.
You either believe them and send the error reports, or you don’t believe them (or for other reasons) and don’t.
An online person(ality?) getting their nose out of joint significantly enough to post on the CIO website that Microsoft didn’t answer his question really induces a big case of “and?” for me I’m afraid.
Stop using the product. If everyone thought that way eventually the Bug will be fixed one way or another. Try downloading Linux or even buy a Mac if you want to purchase something, but please stop whining when you realize that the turd actually does stink.