One of the biggest problems with the HTML standards is that there are two different groups that need standards for HTML and requirements of the two groups are different.
The two groups are those who create the browsers themselves and those writing web pages to display in those browsers.
HTML 3.2 and the draft HTML 5 are very much aimed at providing the standards needed by those developing the browsers themselves so as to tell them just what tags that the browsers need to be able to understand and what to do when optional tags are omitted. These standards are designed (as they should be for the browser writers) to cover all of the ways that someone might reasomably secide to code their HTML (including those who don't have a clue as to how to do it properly) so as to be able to try to make sense out of what the page contains and do as best a job as can be done to actually work out a way to process it.
HTML 2 and HTML 4.01 are closer to what you'd expect standards for web page authors to be but do in fact still have a significant bias toward the browser writers in that tags which ought to be mandatory are still listed as optional. They do come a lot closer to being appropriate standards for web page authors and those areas where they don't are understandable since the one standard is still trying to fulfil both roles as a standard for both browser writers and web page authors.
The only way that we can resolve this issue is to acknowledge that the two groups have different requirements and that the standards that are currently being developed have the purpose of telling the browser writers what their browser is supposed to support. What we need is a separate standard for web page authors that tells them which tags that the browsers need to support that they should avoid using and what tags the browser considers to be optional that they should consider to be mandatory. There are a huge number of things that the standards allow (and which web browsers need to support) that make no sense whatever when you are trying to create a well structured easy to maintain web page.
Currently the closest alternative that we have to a useful standard for web page authors trying to write HTML is in fact the XHTML 1.0 standard which at least identifies the HTML tags that should be considered as mandatory by web page authors correctly. It is for this reason that some web page authors use an XHTML doctype on their web pages even though they are serving the page as HTML and not XHTML.
We really need a way to go one step further than that though. What is really needed is a validator that will correctly identify not whether the page source is valid according to a particular HTML standard but which goes further and advises whether the page is validly coded uasing just that subset of the standard that the browser writers have that is also appropriate for use by those actually trying to create properly coded web pages.
For example, the HTML standards say that <head> and <body> tags are optional but by omitting them you make your web page harder to maintain. Similarly </p> and </li>closing tags are also marked as optional but you are only making the page harder to maintain by leaving them out.
Currently, those experienced at writing HTML know what tags they should always include and know what tags and attributes to avoid using and each author has their own informal standard that they follow in writing their web pages. A common standard that all can follow would make things easier though since if it were possible to validate to that standard then making sure that web pages are correctly coded would be easier, those aspects that some web page authors had overlooked in their private standard would be available, plus it would be easier for newbies to move toward a more appropriate way of coding their HTML as such a validator could easily highlight those things that the newbie is using which the web browsers support but which are just going to make the page harder to maintain.
This article written by Stephen Chapman, Felgall Pty Ltd.