I am on the brink of almost considering leaving the Information Technology field (so, not too close, really). I'm sick of not being able to find answers about software issues. The only way to find these answers is guesswork, trial and error, and downright scientific experimentation.

For example: When using Webalizer to monitor website stats, PDF files are included in the list of top accessed files on the site. But they are ranked by "hits" (also listing "kilobytes"). How can I convert "hits" to a useful metric like "page views"?
Does each PDF page count as one hit? Or can a page cause multiple hits, thanks to "byte range requests" breaking up the file? Seems like simple questions. Both Acrobat and Webalizer have predictable behavior which was created by humans. Yet, no websites have any info, Google Groups has nothing, and I probably won't hear back from the Webalizer author or from my Groups post. So, I'll never know how a huge portion of my website is being used.

Another example: What security settings I can apply to a PDF but still allow Google to index it, create a "View as HTML" cache, and/or extract the title and meta info? Can I specify information for the meta info that shows in search results?
I've searched the web and Groups for far too long, finding nothing. I've downloaded numerous PDF files (and observed my own) to see how Google handles different meta info and security settings. Yet I can find no pattern and no answrs. Both Acrobat and Google are finite systems with logic and algorithms which should be predictable and produce repeatable results. Yet they are acting as wild as nature, and the PDFs on my website still look like crap in Google's eyes.

Countless more examples: The countless error codes for countless, random, unreproducable errors that I receive from Microsoft software which are not documented anywhere. They are errors, they have codes, but they are not explained. It's beyond frustrating to hear "My computer crashed for no reason, here's the error I received." and have to reply "That could be caused by a hundred different things and it is impossible to diagnose." No MS knowledge base entries, no Groups answers, and no $200 to pay an MS tech to tell me I need to FDISK.

I don't understand. I'm not dealing with the unknowns of sub-atomic physics or brain chemistry. These software interactions should be possible to determine, and thus, could be documented or answered when asked. These answers must be out there, but I just can't find them. And high level functions at my work are suffering because of it.

I must not be cut out for Information Technology. I must just be too obsessive. That, or it should be renamed to Information Science for all the experimentation that's required to get anything to work. Thank heaven for the kind souls who continuously post their findings to Newsgroups, because without that, Microsoft Office would have killed me years ago.

Am I wrong? Am I nuts? Should I quit my job and move to Colorado to take up professional rock climbing? Or should I go back to my test network and keep testing, testing, testing?

On the errors front - that'll be Microsoft, dude. It is their way

<hops off soapbox>

Welcome to the world of closed-source software. There are many reasons that people like open-source, but one of the big ones is that if you're having this problem with it, that just means that you're lazy.

I recognize that not everything is open-sourceable. Google is a good example, though I'm surprised that they don't document their PDF-scanning stuff better. But if you had the source code to Webalizer, you could just see exactly what it's doing, and change it if you didn't like the way it was doing it.

I'm with you, FireFox.

Everything you cite has a reason. And I don't like the reasons. Most of them all boil down to: Software is written by programmers. In most cases, programmers are a) inherently lazy, and b) managed by idiots.

Well, one comment. Big IT shops usually have standardized "images" for their regular (non-technical) employees. They would not even troubleshoot the problem. They would back up the data (if the non-technical employee is lucky and respectful), burn the image back in with ghost or some homegrown variation and hand the machine back to the user. Done. NEXT!!!

Poor documentation is always bad, but the point of the argument was "my closed-source application has poor documentation". He has no recourse. If he was using an open-source application, he could figure it out, if he really wanted to expend the effort, even if the documentation was just as poor. Extremely shitty is still better than completely useless.

I've found groups.google.com (former dejanews) to be the best resource for finding that which ain't in manuals. I'm surprised how often it works.

I was using Pagemaker (the absolute all time worst program I've ever tried to use, now use Quark) and found that bitmaps with transparent backgrounds, even tho the manual said they should be transparent, aren't in PM documents. Adobe support had no idea what to do (they never do). I used groups.google.com and found a post from 4 years earlier that said if one rotates the graphic from 0 (0.1 degree is enough), the transparent background works.

Information superhighway my ass. More like information super landfill. What you want is probably there, but it takes a lot of digging through rubbish to find it. And like a landfill, you're probably gonna find a lot of interesting things along the way that you don't need. I was looking for three phase chopper controllers (I didn't want something as complicated as a veriable frequency drive) and one of the first hits was about a device called a chopper that's in sirens. unfortunately, I spent the next six hours reading about air raid and fire sirens. Really interesting stuff. I didn't find the 3 phase chopper I was looking for until the next day.

Sadly, days when programming was at least attempted to be treated as a well structured engineering discipline are long gone. Today we are bombarded with new "paradigms", new "frameworks", new reinvented wheels every day, all hastily clobbered together as if nothing before them existed. Good engineering does not work that way. Innovation is neccesary (not appearance of innovation), but so is building upon prior art. If engineers built bridges the way software is constructed, they would all be beautiful, looking as if they came from a fairy tale or Star Wars cities, but would collapse with the first bird alighting on them...

In other words, I am with you.

I agree that open source offers, at least theoretically, a way out of a situation like the one you describe. However, I will take this as the opportunity for a bit of a tangential rant on my pet peeve about most of OS development:

There are hundreds of simple text editors on sourceforge. There are countless application development frameworks, web UI toolsets, data abstraction or persistence layers... Most of them unfinished. Is the author of text editor #1001 certain that his ideas are so radical that the best way to express them is to write a totally new program, instead of improving one of previous 1000? Another symptom of this is a typical description of version n+1 of many things: "Total rewrite of version n". Don't people design things any more!? (Countless books on refactoring, agile methods, extreme programming ets do seem to discourage design and thinking things out before starting to bang away on the keyboard. Other source of poor software is opposite: slavish adherence to "design patterns" without really understanding them... )

How many universally useful, widely accepted and relied upon OS products are there: Linux itself, Apache (but not all of their countless, short-lived sub-projects), PostgreSQL and MySQL, Perl, Python and PHP, JBoss, perhaps a dozen more. Pretty little, given enormous talent and time that goes into it. Granted, there are zillions of particular, niche itches that got succesfully scratched by efforts of most often a single person (like our Hijack, jEmplode and others), but still...

Please, OS hackers, devote your time and energy to quality, not sheer quantity of software!

Software patents make using prior art a bit of a PITA. It's definitely one of the reasons for so many different wheels.

You do bring up good points though. I'm about to start looking for a C/C++ programmer and I'm trying to get someone who is relatively fresh out of school. First so I can get someone that hasn't already picked up too many bad habits - someone that will be able to move into our own stream of development and quality-focus. At the same time, I'm only interested in the brightest candidates and little previous job experience lessens the amount of previous work I can evaluate.

Well written and plentiful source documentation (as well as detailed design notes) are an absolute must. Having someone else be able to jump into an existing project quickly is a very valuable asset.

Bruno

Bruno

How many programmers think of poor documentation as a form of job security?

Argh, OK, that "dozen" was not fair, but it is still frustrating how many promising projects get abandoned while so much effort is endlessly duplicated.

If we had funding, my tiny company would retain you or someone else willing and knowledgeable to move Apache XML-FOP from version 0.20.5 it has been stuck at for the last two years. Of course, it has been merged with another Apache project, then completely redesigned... They do promise 1.0 before the end of the year.