tag:blogger.com,1999:blog-295836992024-03-14T14:50:20.499+11:00SteveJ-on-ITThirty Years in I.T. Theories, Ideas, Opinions....
Leveraging knowledge of the past to understand now.
@SteveJCbr & stevej.cbr@gmail.comsteve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.comBlogger195125tag:blogger.com,1999:blog-29583699.post-70225523349147350112022-04-24T16:57:00.004+10:002023-08-07T17:22:46.055+10:00John Lions Unix Commentary: Tanenbaum on public record noting 'book' was crucial to MINIX, leading directly to Linus, Linux, iOS/MacOS and Android.<p>To paraphrase Tanenbaum: "No Unix, No Lions Commentary, No MINIX, No Linux, No Android, No iOS/MacOS". </p><p><a href="https://drive.google.com/file/d/1O-PCRAqvlm27Qp-SHgebh8O8bnL9dcOR/view">Text file with links, commentary & edited video transcript</a></p><p>Andrew Tanenbaum, creator of MINIX, gave a talk at the <a href="https://www.youtube.com/playlist?list=PLBWflAwPcgSfCB4acNSxe7G6fDyaH2OOT">UNSW John Lions Distinguished Lectures </a>day on 29 July, 2021, which for copyright reasons wasn't uploaded.</p><p>A comment has been added to the "<a href="https://www.youtube.com/watch?v=pyuxggu193A&list=PLBWflAwPcgSfCB4acNSxe7G6fDyaH2OOT&index=1">Welcome</a>" video with links to the 2016 CACM article and MINIXcon talk included in the text file linked above.</p><p>Tanenbaum notes that AT&T changed the license for Unix V7 and after to ban books or teaching the source code. He was forced to develop MINIX, a functional clone of V7 and free of AT&T "IP", in order to again teach Operating System <i>from the source</i>, not theory only.</p><p>Tanenbaum wrote the O/S on and for an IBM PC, affordable for students, then wrote & published his book, with full source on floppies. In 1991, Linus bought a PC and began using & studying MINIX.</p><p>Linus then used MINIX as the development environment for early versions of Linux.</p><p>"Open Group", the current owners of the Unix trademark, have a "<a href="https://unix.org/what_is_unix/history_timeline.html">History and Timeline"</a> page.</p><p>One of the best & most complete charts of the <a href="https://www.levenez.com/unix/">Unix Timeline is from Levenez</a>. [Multiple PDF versions]</p><p>A much more complete history of Unix variants at Bell Labs and their descendants in the first 10 years by Warner Losh is "<a href="https://papers.freebsd.org/2020/FOSDEM/losh-Hidden_early_history_of_Unix.files/slides.pdf">Hidden Early History of Unix</a>" [PDF] from the "50 years of Unix" effort.</p><p><br /></p>steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-53207974550644959772021-02-02T16:16:00.001+11:002021-02-03T13:45:07.785+11:00Evolution of Hyper-partisanship in Media & Politics<h2 style="text-align: left;">Thesis</h2><p>Have we just seen the End Game of US Capitalism and Democracy with the Republican Party <i>not</i> prepared to criticise, condemn or call to account a President who actively sought to overthrow a pillar of Government and backed an attempt to <i>decapitate</i> the Congress, to have killed the whole of the elected line of succession?</p><p>Elsewhere, this would be a grand Coup, with the perpetrator immediately removed from power and imprisoned. In sections of the US media, this is lauded and applauded, the antithesis of Democratic principles and responsible reporting.</p><p>The problem with the "American Disease" is that "where the USA goes, the world follows". No country is immune from copycats or the consequences of changes in US politics.</p><p>We didn't get here by accident. This was predictable, preventable and foreseeable, a consequence of well known, well tested principles.</p><p>For a century of more, our brains and psychology have been consciously exploited and used against us by those who'd you hope might have our interests at heart, but sought power or fortune for themselves and associates.</p><p>Social Media and new Media via the Internet have increased the power of this exploitation many fold, resulting in dysfunctional Government in many Western Democracies and the destabilisation of the largest, the USA.</p><h2 style="text-align: left;">How we 'work'</h2><p>Our Brains / Minds are wired for efficiency and speed: we filter everything we perceive and think through the filters of our Beliefs then Knowledge. To quote <a href="http://www.vpri.org/pdf/Kay_How.pdf">Alan Kay</a> [pdf], "If you don't believe it, you can't see it."</p><p>The 'unvarnished' Truth is <i>always</i> uncomfortable because it challenges this filtering: it jibes with our view of the World and our brains <i>need</i> to resolve the conflict. This internal conflict is "<a href="https://en.wikipedia.org/wiki/Cognitive_dissonance">cognitive dissonance</a>", a powerful agent of change, and is resolved in many ways - from attack & denial, ignoring, or 'partitioning' our world view to acceptance and 'integration' into a persons' belief system, knowledge or world view.</p><p>All of these approaches are <i>adaptive</i> and <i>successful</i> to some degree. The proof is Social & Genetic "Selection": more functional adaptions, genetic, mental or social, by definition, <i>survive more often,</i> while less functional, or wholly dysfunctional, adaptions <i>lead to lower rates of survival</i> and possibly extinguishing the trait or approach. Skills / behaviours that lead to an early death cannot be directly inheritable.</p><p>Deep Change is <i>hard </i>and<i> challenging</i> because we need to change our view of the World and, quite probably, our Beliefs as well.</p><p>People routinely <i>die for their Beliefs</i>, not just in Wars, Protests or Imprisonment, but in much more mundane ways: on the roads, in hospitals, on the streets, at work and at home.</p><p>The easiest example is the number of people addicted to legal drugs / activities - smoking, drinking, gambling - who would, and do, rather <i>die</i> than <i>Change</i>. Cancer & Cardiac wards see many deaths of smokers who would, or could, never quit.</p><p>The notion that we are all wired, at a basic level, to fight for survival, <i>no matter what</i> is only partly true. This primal survival instinct is surprisingly strong for those who've not experienced it themselves.<br />There's many people who know they need to change to improve some aspect of their life, even exactly how their 'choices' are destroying their lives or killing them and how they need to change to survive, yet choose to continue along the same path.</p><p>The reasons are always complex, but always revolve around being <i>unable</i> or <i>unwilling</i> to change.</p><h2 style="text-align: left;">Political Polarisation and Hyper-Partisanship</h2><p><i><b>You're with me or against me</b>: the rise of polarised, Black & White arguments and positions.</i></p><p>The Canadian, <a href="https://en.wikipedia.org/wiki/Marshall_McLuhan">Marshall McLuhan</a>, studied the interaction of Media on Society and vice versa, in the late 1960's formulating the maxim, "The Medium is the Message".</p><p>First came the Polarisation of the Media with the normalisation of extreme views and hyper-partisanship involving abuse, disrespect, denigration and high-emotion instead of reasoned debate, listening to other points of view and working hard to find common ground or workable solutions.</p><p>Politics became polarised and partisan because of the Media and the Media fed off the Political conflict and partisanship. This created a vicious circle, where the Media and Politics reinforced each other, feeding back via isolated "echo chambers" where people were only ever exposed to <i>one</i> side of an argument, inveigled to join 'a tribe' who "all thought the same (correct) way" and attacking anyone outside 'the tribe' as wrong, ignorant or uninformed.</p><p>This march to extreme, entrenched & intractable positions, sometimes on both sides, was slow, incremental and unstoppable because of the underlying principle of <i>difference</i>, not <i>uniting</i> / <i>commonality</i> - this was the break from the past.</p><p style="text-align: left;">Nations in the Western Democratic tradition are <i>Common-Wealths</i>:<br /></p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><p>they exist to increase the <i>Common</i> <i>Wealth</i> of <i>everybody</i>, not the few, not an elite, not just one 'tribe', group, ideology or political party. Democracies are run in the interests of <i>everybody</i>.</p></blockquote><p>American Politics have had a guiding principle:</p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><p style="text-align: left;"><i>National Interest </i>before<i> Party Interests </i>before<i> Personal Interests.</i></p></blockquote><p style="text-align: left;">This principle has been steadily undermined, forgotten and abandoned over the last 5 decades. To the extent that a President can, and has, without <i>censure or consequence</i> ignore not only all normal conventions, but black-letter law that limits his powers and actions meant to ensure public office and the public purse aren't abused and misused.</p><h3 style="text-align: left;">The role of 'the Internet' in creating extremists</h3><p>With the rise of the modern, fast, "always connected, anywhere" Internet since 2000, the term "self-radicalised" has come into being and is used more frequently, no longer restricted to militant and extreme non-Christians, but now "home grown" white, Christian terrorists.</p><p>It's not <i>The Internet</i> that radicalised people, it's <i>Social Media</i>, specifically the "<i>Algorithms</i>" used select what is included or excluded in users "feeds" or offered as selections.</p><p>There are two interlocking feedback mechanisms at the root of how Social Media select what users see:</p><p></p><ul style="text-align: left;"><li><i>including</i> "more of the same", expressed as "you've liked these in the past, here's more similar", </li><ul><li>whilst <i>excluding </i>material and views <i>not</i> consistent with a user's "likes", "interests", or "preferences", usually consistent with their online social connections, directly leading to "echo chamber", or "like minded people" effects.</li></ul></ul>and<br /><ul style="text-align: left;"><li>increasing "<i>engagement</i>", increasing how often people 'like', 'follow' or view items in their feed.</li><ul><li>People "engage" more when they react emotionally, not logically, and engage most after having a highly emotional response.</li><li>This leads to a slow and inexorable march to being shown more and more extreme views and material in their feed.</li><li>As a person normalises a sets of views / material, their emotional reaction to those items diminishes: Social Media firm can only generate more strong responses by increasing sensational, shocking or "outrage" provoking items in a users' feed.</li><li>From decades of TV studies, before age 10, most children are desensitised to violence and murder. The gaming genre of "First Person Shooter" amplifies this desensitisation.</li></ul></ul><p>People become <i>addicted</i> to Social Media, suffering pain, anguish & other symptoms on <i>withdrawal</i>, as well gaming and other Internet Media as new Media firms better understand our Psychology and provoke automatic responses. <i>The extreme cases are people who die while binging or engaging on-line.</i> There's a growing genre of "the last selfie" as people are accidentally killed trying to capture their "ultimate" selfie.</p><p><i>This is deliberate behaviour with intentional effects by Social Media companies</i>, designed to <i>manipulate</i> us into spending more time with them, consuming more of <i>their</i> content (because <i>they</i> assume copyright of the images, videos and text users post), giving up more information about ourselves and, finally, generating more revenue for them.</p><p><i>This should not be shocking to any adult.</i><br />This has been the modus operandi of all Advertising, especially TV, and Casinos & gambling / gaming venues for most of the 20th Century. Gambling addicts don't <i>just happen</i>, they are wilfully created by Gambling venues by preying upon ordinary people when they're susceptible.</p><p>"Qanon" and the 6-January insurrection & invasion of the Capitol Building were directly mediated and created by Social / Online Media. The impact on, and threat to, Western Democracies is no longer theoretical or improbable: in military terms, this is a Clear and Present Danger.</p><p>McLuhan would not be surprised at either the power of New Media or the outcomes from it's unregulated use.</p><p>There are very strong arguments to regulate Social / Online Media, Gaming Industry, Advertising & Gambling Venues to put limits on how, and how much, they can prey on customers.</p><p><i>The Public Interest</i>, especially in Western Democracies, should be mandated as <i>coming before all commercial interests</i> or we will have exponentially increasing dysfunction, such as the Insurrection, leading to the end of Civil Society. Literally, our survival depends on it.</p><p>This same principle should apply to Food (Obesity & Diabetes epidemic), the Environment (pollution, habitat destruction, species extinction), Sustainable Energy, Manufacturing and Transport, Safe Housing and Healthcare. In our Age of Abundance, is there any reason for anyone in a Western Democracy to live in poverty or with a treatable illness?</p><p>Critically, <i>where is the line, </i>where does <i>our responsibility to our Fellow Man </i>end? Is it on arbitrary national borders, geographical boundaries or what?</p><p>Who can offer a limit on others that they'd be happy to have imposed on themselves? This is the essence of Christian Doctrine, at the heart of Western Democracies: <i>Do unto others as you'd have them do unto you.</i></p><p><i>Free Speech</i> and <i>Free Markets</i> may be basic Rights and may be the cornerstone of our financial and economic systems, but they are not, have never been without limit or Responsibility.</p><p><i>If the Public / Common Interest doesn't come before all political, economic and commercial interests, then what does?</i> Who are the privileged few who we are going to gift unlimited benefits? On what grounds do they deserve this gift?</p><h3 style="text-align: left;">Politics</h3><p>Trump in the USA and BrExit in the U.K. can be traced directly to hyper-partisanship in Politics led by the the Extreme Right capturing the Conservative right and leading to escalating and complementary responses from other groups, even some Progressive / liberal groups.</p><p>"Political Debate" has devolved to personal attack, shouted abuse, paranoia & populism, unabashed lying, denial, deflection, distraction and "the Big Lie". The zenith demonstrated first by Trump winning the Republican Party nomination, then his election campaign leading to uncontrolled, escalating behaviours as 45th President.</p><p>New double-speak phrases arose in this fact-free, unaccountable world: "Alternative Facts", "Fake News", "Enemy of the People", "Very fine people on both sides", "people are saying"</p><p><a href="https://en.wikipedia.org/wiki/FCC_fairness_doctrine">FCC "Fairness Doctrine"</a> est 1949, abandoned in 1987 under Reagan & his "Deregulated Free Market" approach, leading in part to "<a href="https://en.wikipedia.org/wiki/Political_polarization">Political Polarisation</a>", due to a perceived“chilling effect” upon Freedom of Speech with Cable & Satellite networks claiming exemption. <a href="https://www.usatoday.com/story/news/factcheck/2020/11/28/fact-check-fairness-doctrine-applied-broadcast-licenses-not-cable/6439197002/">Like many things, it's a complex issue</a>.</p><p></p><ul style="text-align: left;"><li>required broadcast licensees to both present controversial issues of public importance and</li><li>to do so in a manner that was honest, equitable, and balanced.</li><ul><li>with subjects of Editorials, or perceived unfair attacks, given the Right of Reply.</li><li>candidates for Public Office given equal air-time.</li></ul></ul><p></p><hr /><p></p><p>Schooling: without fluency in the current Information Tools, how are people supposed to use them well and wisely. To avoid falling for conmen and charlatans, to discern real vs 'fake' news or information?</p><p>The failure of general US public school system to teach well and prepare young people for their adult lives, directly gave rise to the current Polarised, hyper-partisan culture and politics.</p><p>Qanon: "Research it for yourself": False premise, these people know how to do good 'research'.</p><p>but if people aren't taught at school <i>how</i> to do research, how to critically assess what they see and hear, and importantly the "quality" of the source - </p><p>=> Fox News, Pushing the extreme right political & economic ideology of Murdoch. Amplified by other media reporting their most outrageous comments. 1.5M subscribers and $1B revenue - with a much larger impact.</p><p>-> Trump's autocratic, demagoguery accepted, amplified and normalised on Fox News.</p><p>=> Tabloid Journalism and "Race to the Bottom": Sensationalism, pandering to prejudice, fears & paranoia of audiences.</p><p>=> Republican Party has been corrupted by hyper-partisanship and holding onto power with Senate being a road-block to legislation for decades: "Power at any Cost, Unquestioning Loyalty to Ours, Uncritical Demonisation, attacks and denigration of 'Others'" demonstrated by their unflinching resolve to not impeach #45 regardless of evidence (Russian 2016 Interference) or unequivocal incitement to attack them personally with the possibility of decapitating the entire Congressional <a href="https://en.wikipedia.org/wiki/United_States_presidential_line_of_succession">Line of Succession</a>.</p><p>Normalisation of "Common Sense" and "You just know it's true", alongside distrusting and questioning Expert Opinion and Advice, leading to rejection of the Science of Climate Change without rational cause.</p><p>People are proudly "Anti-Expert". What the?!?!?!</p><p>The mass perversion of "Christianity". "The <i>love</i> of money is the <i>root</i> of all Evil".</p><p>The "Great American Dream", the "<i>opportunity</i> for prosperity and success for everyone" has devolved into greed and a desire for riches, leading to just three of 300 million people controlling or owning half the assets in the USA, and one thousandth of the population owning 80% of its assets.</p><p>For a notionally religious, Christian country, this state of affairs where "avarice beyond dreams" is accepted and unremarked seems contradictory.</p>steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-79970467733359270552021-02-02T09:35:00.003+11:002021-02-02T09:35:41.185+11:00Why FOSS: standing on the Shoulders of GiantsIn response to a great run-down of <a href="https://cameronshorter.blogspot.com/2016/11/the-elusive-open-business.html">"Why FOSS" - standing on the Shoulders of Giants</a>.<div><br />
Loved the Daniel Pink video: life & work aren't only about Profit at any cost.<br />
<br />
longer version at:<a href=" http://stevej-on-it.blogspot.com/2016/11/why-open-source-google-facebook-amazon.html"> http://stevej-on-it.blogspot.com/2016/11/why-open-source-google-facebook-amazon.html</a><br />
<br />
Humans in groups benefit more from general co-operation than competing fiercely all the time.<br />
Competition creates 'challenge' and prevents complacency and laziness - it's not a bad thing in itself.<br />
<br />
But competition comes with downsides:<br />
it's economically 'inefficient', re-inventing the wheel repeatedly.<br />
In Engineering terms, everyone reinvents the wheel, repeating some, not all, mistakes<br />
and no one group ever learns all the 'good' things. Until they merge or are bought up...<br />
<br />
Monopolies, or Oligopolies, actively suppress competitors, reduce diversity and multiply costs to consumers.<br />
They may be 'private', but they are anti-competitive and economically disastrous for the markets they hold captive.<br />
Think "Too Big to Fail" and the 2008 global financial crash to understand it's more than just goods affected.<br />
<br />
Open Source actively prevents the worst outcomes & encourages the best that Capitalism has to offer: competition of ideas, sharing of Information and knowledge and building on the work of others.<br />
<br />
Open Source scales, encourages invention, diversity, novelty & rewards innovation, hard work & risk taking.<br />
It actively prevents asymmetric rewards and 'winner takes all'.<br />
<br />
This is capitalism in action. It's mass competition within truly free markets.<br />
<br />
Open Business _is_ risky: you can't be complacent or stop Putting the Customer First.<br />
A simplistic analysis might brand Open Source "socialist", but it is the precise opposite.<br />
Open Source underpins fierce, wide-spread competition, while allowing customers needs to be met.<br />
Open Source truly levels the playing field, allowing anyone to earn money based on merit not an inherited position or privilege.<br />
<br />
Open Source is an unfamiliar model to many:<br />
it's fierce competition, that embraces sharing, egalitarian market access and equitable rewards for work.<br />
<br />
It's the Epitome of Adam Smith's Free Market, not anything else.<br />
Sharing source code removes information-based market distortions: all sellers and buyers have full market information.<br />
<br />
Google, Facebook and Amazon got huge on the back of Open Source and actively contribute their work back.<br />
Notably, not _all_ their work, but key parts.<br />
<br />
These big Internet companies have huge turnovers, have made their shareholders rich and are known for being fiercely competitive. Their success is based on Open Source & they know this: actively supporting developers & projects.<br />
<br />
To characterise their commercial success as solely due to Open Source, or within the reach of every 2-person start-up is wrong. For every super-large Open Source business, there are tens of thousands of failed startups, thousands of short-lived businesses, hundreds of sustainable mid-scale businesses and a large community of individuals that share and benefit in the work over time.<br />
<br />
Open Source is driven in the short-term by profits or altruism funded by research grants or personal interest and exertion.<br />
<br />
In the long-term, we know from Google, Facebook et al (& foundations like 'Apache' and 'GNU'/'FSF'), that money is the fuel for Open Source development: and it's mainly contributed 'in kind' by big corporates acting in their own commercial interests.<br />
<br />
IBM, HP, Redhat & Canonical/ubuntu, even Intel and Microsoft, collectively spend billions each year on software & testing they seemingly 'just' give away. For them, this is not philanthropy, it's a hard-nosed business investment that repays itself many-fold.<br />
<br />
If large Corporations at the heart of the US economy think Open Source makes financial, business and marketing sense, then what is everyone else missing?<br />
<br />
Money makes the world go round and Open Source generates stunningly large sales and profits for those smart enough to embrace it and realise it confers a competitive advantage.<br />
It's just good business.</div>steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-16569104066718325602021-01-31T20:24:00.004+11:002021-02-02T09:10:07.642+11:00Scott Morrison vs Google<div>Let's not beat around the bush, this latest draft legislation is a "<i>Fox News Tax"</i>, nothing less.</div><div><br /></div><div>Consider:</div><div><ul style="text-align: left;"><li>The Flow of Value</li><ul><li>Google, even with 96% share of "Search", gains <i>no</i> revenue from collecting, indexing, curating & presenting "news" content. It's a Free service, a true 'loss leader'.</li><li>Aus Media sites, esp newspapers, gain significantly from the massive amount of traffic flow ('clicks') driven to their sites.</li></ul></ul><ul style="text-align: left;"><li>Google & Facebook do <i>not</i> compete in the Advertising segment, Classifieds, where Aus newspapers lost 92% of their revenue since 2002, when Google entered the new segment, online advertising market.</li></ul><ul style="text-align: left;"><li>The impact on Aus Newspapers if Google withdraws from Australia, or simply stops indexing Aus Media sites.</li><ul><li>Newspapers will lose traffic (= page views = revenue) immediately, until perhaps, other search engines take over.</li><li>No Media or newspaper will increase their revenue from Display or Classified Ads if Google withdraws. </li></ul></ul></div><h2 style="text-align: left;">Foreign Political Interference</h2><div>
Rupert Murdoch is a Foreign citizen, yet isn't just allowed to own media all across Australia, isn't confined to one media type, print, he's allowed to dominate - even monopolise - large media markets: News Ltd are the <i>only</i> daily newspapers in multiple Capital Cities and along with Nine-Fairfax come close to an effective media monopoly with 80-90% of readers/viewers in most media markets.</div><br /><div>
Murdoch's holdings alone <a href="https://theconversation.com/factcheck-does-murdoch-own-70-of-newspapers-in-australia-16812" target="_blank">account for at least two-thirds of Capital City and National</a> newspaper "circulation", the most politically influential market.</div><br /><div>
Since 1972, Murdoch's media holdings have taken a vociferously, even violently, anti-ALP stance. In a global media empire with a decidedly Extreme Right bias and a reputation for editors being fired for not toeing "the party line", protestations of "Editorial Independence" are meaningless and hollow. All Murdoch's media holdings operate as his personal mouthpiece, even anonymously publishing his opinions.</div><br /><div>
In his personal tweets, Murdoch is uncompromisingly anti-ALP, anti-"left" / anti-"progressive. The editorial stance of News Ltd and Sky News is consistent with Murdoch's personal views.</div><br /><div>
Murdoch's father, Keith Sr, was a founder of the highly secretive, ultra-Right Wing lobbyist grop, IPA ("Institute of Public Affairs", self-styled as "the voice for freedom since 1943"). Murdoch continues to be closely associated with the IPA, who use their platform to directly, or via the media, influence politicians at every level, even openly "suggesting" 100 policies (<a href="https://web.archive.org/web/20121104070111/https://ipa.org.au/publications/2080/be-like-gough-75-radical-ideas-to-transform-australia/">first 75</a>, <a href="https://web.archive.org/web/20130214031751/http://ipa.org.au/publications/2110/25-more-ideas-for-tony-abbott/">another 25</a>) for Tony Abbott as PM, to reverse Gough Whitlam's policies from decades earlier, now part of bedrock of Australian society. The number of IPA policies Abbott "ticked off" should concern anyone interested in an Open & Transparent Democracy, unpinned by free and fair elections, not run by shadowy, "faceless men" in back rooms, doing deals with mates.</div><br /><div>
There were strict "Foreign Interference" laws introduced for elections, but seemingly don't apply to a Foreigner with effective media monopolies across Australia, who's blatantly and unabashedly pushed hyper-partisan Right-Wing editorial policy for the last 16 federal elections on the trot.</div><div><br /></div><div>Just for good measure, Murdoch's media in Australia pays almost no tax here, alone has had multiple multi-million grants, without public reasons, from the Federal Government to support it. While in 2014, the Abbott Government <a href="https://www.theguardian.com/commentisfree/2014/feb/20/the-end-of-the-age-of-entitlement-doesnt-apply-to-murdochs-empire">directed the ATO to "rebate" News Ltd $882 million</a> after a long running $2 billion law suit.</div><h2 style="text-align: left;">Problems with the Proposed "Tax"</h2><div>The idea of taxing <i>some</i>, <i>not all,</i> online advertising businesses and arbitrarily transferring those taxes to <i>some</i> (large) private companies should ring alarm bells. There is no precedent for this type of anti-"Robin Hood" tax transfer in any Western Democracy.</div><div><br /></div><div>Once this tax transfer mechanism is established, where does it end? Will Gerry Harvey ask for a cut of Kogan, e-Bay and gumtree revenues? </div><div><br /></div><div>How is this even legal, taxing selected entities for the enrichment of particular commercial concerns?</div><div>There seems no basis for selecting recipients, other than "Have a Go, Get a Go".</div><div><br /></div><div>There's two other elements that the current draft legislation does not address:</div><div><ul style="text-align: left;"><li>the mechanism limited to "news" by journalists, does not scale to other content creators who have no means to charge fairly for their work. <i>This should be a universal micro-payments scheme, open to all content creators equally.</i> </li><ul><li>The Copyright Council already collect and distribute fees for use of many types of copyright material. The concept of "fair compensation" for Intellectual Property is already well established & tested, yet ignored by this highly targeted, highly selective and asymmetrical proposal.</li></ul></ul><ul style="text-align: left;"><li>The scheme does not recognise Copyright or help to enforce any sort of Content Classification system. There is no way to discern "original news content" from "newswire material" or "edited media releases", let alone opinion pieces, sports or entertainment commentary. Or various types of restricted material, including pornography.</li><ul><li>HTML has "meta" tags, meant for exactly this purpose.</li><li>The obvious <i>technical</i> implementation is for Content Creators to "tag" their pages, allowing Aggregators and Search Engines to automatically process it, and if required, pay for "using" it.</li><li>The same <i>technical</i> solution of tagging content, especially "Adult Material", would make the operation of a National "black list" trivial - all untagged content is assumed to be "restricted". In parallel, it'd be possible to provide specific User X.509 certificates allowing access to "Adult Material" in Australia.</li></ul></ul></div><h2 style="text-align: left;">Just the Facts</h2><h3 style="text-align: left;">Google didn't steal revenue from newspapers</h3><div>There's any amount of hysterical 'reporting' in the mass media proclaiming, <i>with no basis in fact</i>, that if Google & Facebook don't acquiesce to this tax, <a href="https://www.theguardian.com/commentisfree/2021/jan/22/googles-threat-to-withdraw-its-search-engine-from-australia-is-chilling-to-anyone-who-cares-about-democracy">it will be "the end of democracy" as we know it.</a></div><div><br /></div><div>The economic analysis firm, AlphaBeta, has released a comprehensive report on the <a href="https://alphabeta.com/wp-content/uploads/2020/09/australian-media-landscape-report.pdf">"Australian Media Landscape"</a>, specifically addressing the evolution of Google's on-line advertising versus print advertising from 2002 to 2018, plus 2017-2020 data for preferred method of consuming news.</div><div><br /></div><div><div>Since 2002, when Google entered the advertising market here, the whole market has nearly <i>doubled</i>.</div><div>So <i>what</i> could News Ltd and friends in print media stand to gain, if their business had grown commensurately?</div><div><br /></div><div>Which is the point. </div><div>News Ltd advertising revenue has collapsed and they are looking to the Government to tax Google for them.</div><div><br /></div><div>While the <i>rest</i> of the Australian Advertising Industry has gone ahead in leaps and bounds, <i>newspaper revenues have declined by ~30%</i>.</div><div><br /></div><div>But this wasn't Google "eating their lunch".</div><div><br /></div><div>92%, $1.3B of $1.4B, of the collapse of Australian newspaper revenues is in an advertising segment that Google (and Facebook) don't even participate in: </div></div><div><br /></div><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><div><div style="text-align: left;">Classified Adverts: the sort that individuals use to advertise their car, boat, old computer or house.</div></div></blockquote><p>The same revenue stream that <a href="https://www.theguardian.com/media/2005/nov/24/pressandpublishing.business1">Murdoch himself once referred to as "Rivers of Gold"</a>, yet collapsed under his stewardship.</p><p>Google invented a whole new class of advertising:</p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><p style="text-align: left;">tied directly to the content, <i>selected and served in real-time</i> and often finely targeting the individual.</p></blockquote><p>Mass media - radio, TV and print, by definition "fixed" and "uncustomised", have never been able to come close to this level of targeting or reporting outcomes to buyers of advertising.</p><p>It's irrational and faulty logic to argue that "Google increased their advertising revenue and newspapers lost out,<i> therefore, Google 'stole' their business</i>". That's false attribution.</p><p>In 2000, the mass media companies understood that "The Internet" was going to change Sales and Advertising from traditional "physical" or off-line models. Online Sales have boomed and alongside that, so has the new <i>online</i> Advertising.</p><p>Australia Post has ridden this boom in Online Sales, vigorously competing with other private courier companies. It's proof that "Old" model businesses are able necessarily to transition to new business models.</p><p>In 2000, those mass media companies were flush with cash. They had all the time, money and resources to not just participate in these new markets, but to leverage their existing customer base and their loyalty, into the new on-line world. There is no good commercial reason for Mass Media companies to have not translated to the new, online advertising world and to fail to leverage their existing customer bases.</p><h3 style="text-align: left;">If Newspapers are owed restitution, then by whom?</h3><p>The bias and irrationality of this proposal is clear when looking at the segment that lost everything over the last 15 years: print directories were $1.2B segment in 2002 and have since disappeared.</p><p>Why hasn't the government, Productivity Commission, ACCC or News Ltd & Nine-Fairfax mentioned fair restitution for <i>print directories</i>? If any industry segment is deserving of compensation, it's the one that's been obliterated by online services.</p><p>The AlphaBeta data & analysis clearly shows that Newspapers over the last 15 years presided over the collapse of their own business. They had the Means, Motive and Opportunity to translate from major players in "Old" Advertising to being <i>the</i> major on-line Advertisers.</p><p>The collapse of the Print Media business model <i>is solely due to their failure to compete</i> in an open marketplace. The disruption in their business model wasn't due to a change in licenses, like the Dairy or fishing industries, others ignoring the law, like ride-share, nor leveraging a monopoly, like Windows, but the very definition of competition on a level-playing field. This is no precedent for businesses, large or small, to be c<i>ompensated for failing to be competitive</i>.</p><p>Contrary to assertions in government reports to the contrary, in 2002 Google was <i>not</i> the dominant online advertiser. In 2002, Google was a young upstart business with little revenue and zero profits. Google's business and future in 2002 was anything but certain, against the Big Media Players, who had always been <i>very</i> good investments, with deep pockets and a seemingly certain future.</p><p>It is simply incorrect to characterise Google, who worked hard to attract 96% of search requests in 2018, as a dominant and highly-funded operation in 2002, or even the decade leading up to 2018.</p><p>The majority of lost print media revenue came from the move of Classifieds from print to online. If anyone had the foresight and capability to build a comprehensive Online Classifieds system for Australia, it was the Trading Post and the large print media operations. Yet they sat on their laurels.</p><p>We've seen Shareholder Class Actions for lesser reasons in Australia. Perhaps those who've been suckered by the indolence and incompetence of the Mass Media companies might now take a renewed interest.</p><p>I'm at a loss as to why Print Media, who sat idly by as their self-described "Rivers of Gold" were taken from them by Cheaper, Better, Faster on-line alternatives, are owed <i>anything</i>. Why should Google and Facebook be targeted when they don't even compete in that market segment?</p><div><div>Since when do businesses in a Free Market Capitalist system have the right to a tax or levy on competitors who've beaten them fairly & squarely in the market place?</div><h2 style="text-align: left;">Irony upon Irony</h2><div>It's more than a little ironic that the heroic supported of The Free Market, Rupert Murdoch - as we know from his association and accolades from the IPA, has somehow contrived to create a law that penalises those who win in equitable deregulated market competition and rewards Murdoch's businesses for failing to compete.</div><div><br /></div><div>The ultimate test of the "Google stole my lunch" theory is if they leave Australia, will Print Mass Media fortunes be magically reversed? Absolutely not - because Google & Facebook didn't take their Classifieds revenue.</div><div><br /></div><div>I guess this is the central question to ask Murdoch and his Political Allies:</div></div><div><br /></div><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><div><div style="text-align: left;">What does "the Free Market" mean if the Government can swoop in and tax legitimate winners who's only crime is out competing sluggards, then transferring their hard won profit to those who lost?</div></div></blockquote><p>Google wasn't issuing a threat when it said its business model could not support paying an unfair, unjust and undeserved tax. <i>It was stating a business reality</i>. For anyone without full access to its books and Business Plans to say otherwise is being presumptuous, disingenuous and mendacious.</p><p>Nobody but Google knows their financial position and the strength of their position. Any politician or arm of government to presume this was a "threat", not a statement of fact, is being foolish in the extreme. The stakes for Australian business are very high and should be idly disregarded for some imagined political benefit.</p><p>The only people issuing 'threats with menaces' and attempting to bully others is Morrison and his band of unaccountable ministers.</p><div><div><br /></div></div><hr /><h3 style="text-align: left;">Graphics from AlphaBeta Report</h3><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvsQSKI3Y1SPrSVXpjuS8TGI5RAhj_JCIY_4Zeer5Nl4OUdMo0SIqiXBDa9yAeJshyphenhyphentF40AaabOCcKQOGO6XGD3f27J2D1Ek7vvdokzndj9Wej6EjiatS-FoxLucnwGH7i49jzSA/s1444/AB-2020_1.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1000" data-original-width="1444" height="443" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvsQSKI3Y1SPrSVXpjuS8TGI5RAhj_JCIY_4Zeer5Nl4OUdMo0SIqiXBDa9yAeJshyphenhyphentF40AaabOCcKQOGO6XGD3f27J2D1Ek7vvdokzndj9Wej6EjiatS-FoxLucnwGH7i49jzSA/w640-h443/AB-2020_1.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Australian media landscape trends</span></td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggrv6itADlg-UutRGACfT-KnQfTqPtvWODzBgH3uFmGAdvDdhwecBMcoArdjiZsLKxjBwcEYX52v5tn-qa2s3neReiii0BmHvVyhxRhAyQOqOUgsr4c07HoFcXpD0TbCNxj1ApBg/s784/AB-2020_1a.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="784" data-original-width="673" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggrv6itADlg-UutRGACfT-KnQfTqPtvWODzBgH3uFmGAdvDdhwecBMcoArdjiZsLKxjBwcEYX52v5tn-qa2s3neReiii0BmHvVyhxRhAyQOqOUgsr4c07HoFcXpD0TbCNxj1ApBg/w344-h400/AB-2020_1a.jpg" width="344" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Newspaper revenues have declined</span></td></tr></tbody></table><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPJBR272u-7-YxQT7_wQYDeHx7d7HVvEQXvO2p9CJsH1I3bRNxnucXWPkAsgy68Q0K29426muRCPIrmgYeRmVkoVBpLposkFPAuzHHH57XdfhYoDv_EshnXfRuerS9tVAvjeNofA/s661/AB-2020_1c.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="520" data-original-width="661" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPJBR272u-7-YxQT7_wQYDeHx7d7HVvEQXvO2p9CJsH1I3bRNxnucXWPkAsgy68Q0K29426muRCPIrmgYeRmVkoVBpLposkFPAuzHHH57XdfhYoDv_EshnXfRuerS9tVAvjeNofA/s320/AB-2020_1c.jpg" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Online search revenues have grown</span></td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkv7kHeZv-lpdUrxDi79gFznl_nrKwvXSKbBPK9CfXW3xkiGg3MfkdAhQaX-YRw5Qqd3j0pVRshJZnuIhzWYXvbfBiM87wyQZHUTsXvuOaR7fxn7tSjSrU1nS3O_DNZZnJ4FBi3g/s905/AB-2020_2.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="905" data-original-width="844" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkv7kHeZv-lpdUrxDi79gFznl_nrKwvXSKbBPK9CfXW3xkiGg3MfkdAhQaX-YRw5Qqd3j0pVRshJZnuIhzWYXvbfBiM87wyQZHUTsXvuOaR7fxn7tSjSrU1nS3O_DNZZnJ4FBi3g/w596-h640/AB-2020_2.jpg" width="596" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Australian newspaper revenues 2002-2018</span></td></tr></tbody></table><div><br /></div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrSN3AXmuGrh8Iy_D-HL_jQdxLFTBoypt-epiAAY7QoY8aAvH_4h9kB9R0Od5ndFggkQnX6xpht2Npv63jqg9vpMdCp1EVgJHj0TNOOaArBs-4znVgxpPaCOnWCJXPA2Z6TJ5DYQ/s903/AB-2020_3.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="903" data-original-width="844" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrSN3AXmuGrh8Iy_D-HL_jQdxLFTBoypt-epiAAY7QoY8aAvH_4h9kB9R0Od5ndFggkQnX6xpht2Npv63jqg9vpMdCp1EVgJHj0TNOOaArBs-4znVgxpPaCOnWCJXPA2Z6TJ5DYQ/w598-h640/AB-2020_3.jpg" width="598" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Australian newspaper revenues 2002 vs 2018</span></td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRN2nx-UXmZVGRU_Ut-MNUr343wAl91iq8_nA75lpO4bwH0X2Df5fj0caZAeFjYLA53gnVv3pbGMczkpZtxveTrNrczAvme7429UdhAzicxKsen95h79wJrZDy10IAufAHbYTlIA/s905/AB-2020_5.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="905" data-original-width="845" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRN2nx-UXmZVGRU_Ut-MNUr343wAl91iq8_nA75lpO4bwH0X2Df5fj0caZAeFjYLA53gnVv3pbGMczkpZtxveTrNrczAvme7429UdhAzicxKsen95h79wJrZDy10IAufAHbYTlIA/w598-h640/AB-2020_5.jpg" width="598" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Australian advertising revenues 2002 vs 2018</span></td></tr></tbody></table><div><br /></div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4HuiDARTA54lkTagU25olqiIgd-E-iRUtVoIB4QecXCUShNwnagadsFZSEcl6RCxkaEJ8x9KQTw6b39m11SsVV0_-ZX9e3wbm2-eJScYvHGnRBGAbVpJ4CdffnvgxkUs9m_J6gQ/s900/AB-2020_4.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="844" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4HuiDARTA54lkTagU25olqiIgd-E-iRUtVoIB4QecXCUShNwnagadsFZSEcl6RCxkaEJ8x9KQTw6b39m11SsVV0_-ZX9e3wbm2-eJScYvHGnRBGAbVpJ4CdffnvgxkUs9m_J6gQ/w600-h640/AB-2020_4.jpg" width="600" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Classified advertising revenues, print newspaper vs online</span></td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAUgvcveTpxLNYe4EXXcc9aI_jAH-lfEhJxMM77XCOPMfygbSf9dpDT85MxdLjTSmDkRGqtQaYodGzmTJXVVTw0aeXk5LKNLAAhcXGaf9-5sdiXNcocvi6pBjdNTRwGsVA6Mhzwg/s1334/AB-2020_6.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="910" data-original-width="1334" height="436" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAUgvcveTpxLNYe4EXXcc9aI_jAH-lfEhJxMM77XCOPMfygbSf9dpDT85MxdLjTSmDkRGqtQaYodGzmTJXVVTw0aeXk5LKNLAAhcXGaf9-5sdiXNcocvi6pBjdNTRwGsVA6Mhzwg/w640-h436/AB-2020_6.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Online search, display and classified advertising revenues totalled $8.5B in 2018</span></td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4vF8qJXGKNvDeAci_myqV3KsOBuHUM6l5eAFYU_50GHI4aItb0kyHeefWCjU9wLBdQ3eDuWC5Fhgaynt8_aYF4IXqcGQAajJhdKzcY6HFWa4NLTjD6s25tzM6qlJjxjkLlhAgZg/s907/AB-2020_7.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="907" data-original-width="843" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4vF8qJXGKNvDeAci_myqV3KsOBuHUM6l5eAFYU_50GHI4aItb0kyHeefWCjU9wLBdQ3eDuWC5Fhgaynt8_aYF4IXqcGQAajJhdKzcY6HFWa4NLTjD6s25tzM6qlJjxjkLlhAgZg/w594-h640/AB-2020_7.jpg" width="594" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Online search advertising revenues - estimate of sources</span></td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRIn-59CMWLOfhqJpCCf-tAoLBgz9NsAOj33aN3kZsA9h-LKz7j1wCAJoCXUIorqrgoGFCcAOdop9I0mA8kWzZAZQi90MzBQaq8Hee7zYwMW29V-0aJZBybq7rdHtVdARvS64arg/s908/AB-2020_8.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="908" data-original-width="844" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRIn-59CMWLOfhqJpCCf-tAoLBgz9NsAOj33aN3kZsA9h-LKz7j1wCAJoCXUIorqrgoGFCcAOdop9I0mA8kWzZAZQi90MzBQaq8Hee7zYwMW29V-0aJZBybq7rdHtVdARvS64arg/w594-h640/AB-2020_8.jpg" width="594" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Net readership of Australian newspapers 2002 vs 2018</span></td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6qfp47cgBYf6AsupUIZa67G1_MvGs2xODblP2fDzFiuJaHInBU-NF30cQOwj6Cfp42KmVQ45AN6H7uYeblnVvdbTTl6VmwZFBW9Ew2HDFO8me4hnUCZBcZ-kpoE5YNX048bH8pQ/s909/AB-2020_9.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="909" data-original-width="845" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6qfp47cgBYf6AsupUIZa67G1_MvGs2xODblP2fDzFiuJaHInBU-NF30cQOwj6Cfp42KmVQ45AN6H7uYeblnVvdbTTl6VmwZFBW9Ew2HDFO8me4hnUCZBcZ-kpoE5YNX048bH8pQ/w594-h640/AB-2020_9.jpg" width="594" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="text-align: left;">Preferred method of consuming news in Australia 2017 - 2020<br /></span></td></tr></tbody></table><br /><div><br /></div><div><br /></div><div><br /></div><br />steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-9602073189120492222018-07-23T23:54:00.001+10:002018-07-23T23:54:34.305+10:00FMAA s44, now PGPA s 15, 'proper use': The <a href="https://www.legislation.gov.au/Details/C2004A00675">Financial Management and Accountability Act</a> (FMAA) was replaced by the <a href="https://www.legislation.gov.au/Details/C2014A00062">Public Governance, Performance and Accountability Act</a> (PGPA)<br />
<br />
Department of Finance administers the Act.<br />
<br />
<a href="https://www.finance.gov.au/resource-management/pgpa-glossary/proper-use/">Resource Management glossary - proper use</a><br />
<br />
<blockquote class="tr_bq">
<div class="block block-system block-main block-system-main odd block-without-title" id="block-system-main" style="font-family: Arial, sans-serif; font-size: 13px; padding-left: 0px;">
<div class="block-inner clearfix">
<div class="content clearfix">
<div about="/resource-management/pgpa-glossary/proper-use/" class="node node-pgpa-glossary node-published node-not-promoted node-not-sticky author-seanmcdermott odd clearfix" id="node-69751" typeof="sioc:Item foaf:Document">
<div class="content clearfix">
<div class="field field-name-body field-type-text-with-summary field-label-hidden" style="background-color: #e7e7e7; border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; border-top-left-radius: 4px; border-top-right-radius: 4px; border: 1px solid rgb(160, 160, 160); font-family: "Times New Roman", Times, serif; padding: 16px; position: relative;">
<div class="field-items">
<div class="field-item even" property="content:encoded">
<div style="margin-bottom: 15px; margin-top: 15px;">
Efficient, effective, economical and ethical use or management of public resources.</div>
<div style="margin-bottom: 15px; margin-top: 15px;">
For the accountable authority of a non-corporate Commonwealth entity, proper use and management of public resources means behaving, taking action and making decisions in a way that is not inconsistent with the policies of the Australian Government in accordance with sections 15 and 21 of the PGPA Act.</div>
</div>
</div>
</div>
<div class="field field-name-field-glossary-terms field-type-entityreference field-label-above" style="margin-top: 25px;">
<div class="field-label" style="font-size: 20px; font-weight: bold;">
Related glossary terms</div>
<div class="field-items" style="margin-left: 30px;">
<div class="field-item even" style="display: list-item !important; list-style-type: disc !important;">
<a href="https://www.finance.gov.au/resource-management/pgpa-glossary/economical/" style="color: #40807f;">economical</a></div>
<div class="field-item odd" style="display: list-item !important; list-style-type: disc !important;">
<a href="https://www.finance.gov.au/resource-management/pgpa-glossary/effective/" style="color: #40807f;">effective</a></div>
<div class="field-item even" style="display: list-item !important; list-style-type: disc !important;">
<a href="https://www.finance.gov.au/resource-management/pgpa-glossary/efficient/" style="color: #40807f;">efficient</a></div>
<div class="field-item odd" style="display: list-item !important; list-style-type: disc !important;">
<a href="https://www.finance.gov.au/resource-management/pgpa-glossary/ethical/" style="color: #40807f;">ethical</a></div>
</div>
</div>
<div class="field field-name-field-related-pgpa-act-section field-type-entityreference field-label-above" style="margin-top: 25px;">
<div class="field-label" style="font-size: 20px; font-weight: bold;">
Related PGPA Act provisions</div>
<div class="field-items" style="margin-left: 30px;">
<div class="field-item even" style="display: list-item !important; list-style-type: disc !important;">
<a href="https://www.finance.gov.au/resource-management/pgpa-act/8/" style="color: #40807f;">8. The Dictionary</a></div>
<div class="field-item odd" style="display: list-item !important; list-style-type: disc !important;">
<a href="https://www.finance.gov.au/resource-management/pgpa-act/15/" style="color: #40807f;">Section 15. Duty to govern the Commonwealth entity</a></div>
<div class="field-item even" style="display: list-item !important; list-style-type: disc !important;">
<a href="https://www.finance.gov.au/resource-management/pgpa-act/21/" style="color: #40807f;">Section 21. Non-corporate Commonwealth entities (application of government policy)</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="block block-views block-last-updated-block block-views-last-updated-block even block-without-title" id="block-views-last-updated-block" style="font-family: Arial, sans-serif; font-size: 13px;">
<div class="block-inner clearfix">
<div class="content clearfix">
<div class="view view-last-updated view-id-last_updated view-display-id-block view-dom-id-dc286945aeb160528ef63e2c76295fee">
<div class="view-content">
<div class="views-row views-row-1 views-row-odd views-row-first views-row-last">
<div class="views-field views-field-changed">
<span class="field-content"><div style="margin-bottom: 15px; margin-top: 15px;">
Last updated: 13 January 2016</div>
</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-73178859850772758002016-11-27T15:30:00.000+11:002016-11-27T15:30:30.566+11:00Why Open Source? Google, Facebook, Amazon & Netflix can tell you: makes you money.A long comment I had to cut down in response to <a href="https://cameronshorter.blogspot.com/2016/11/the-elusive-open-business.html">this post on Open Source</a>. Highly recommended.<br />
<hr />
Cameron,<br />
<br />
Very good run-down of 'Why FOSS" - standing on the Shoulders of Giants.<br />
Where bugs become either a Shared problem or "Other People's", not a killing 'tarpit'.<br />
<br />
Loved the Daniel Pink video:<br />
we're not just about Survival and Profit at any cost.<br />
<br />
<br />
<a name='more'></a>A Lawyer brother of a mate always questioned his FOSS work:<br />
Why give your work away? [meaning, "you could make so much more if you charged a lot for it"]<br />
<br />
This question is based on both false assumptions and a limited model of economics and business.<br />
<br />
Imagine if Maths, Physics, Chemistry, Biology and other Research only worked this way.<br />
Would you pay 10cents every time you used Pythagorus?<br />
Or would you just 'approximate' Pi to 3? (and see everything you built fall down)<br />
<br />
The 'drag on the economy' of paying for everything would be horrendous.<br />
If we elected to uniformly allow inventors, not the select few, to charge for their work, the global economy would fail.<br />
<br />
Humans in groups benefit more from general co-operation than competing fiercely, in all endeavours. It's why we pay taxes - to share the costs of expensive things that provide disproportionate benefits to us collectively.<br />
Imagine paying 'what the market would bear' for clean drinking water, even clean air. No economy could survive that.<br />
<br />
Competition creates 'challenge' and prevents complacency and laziness - it's not a bad thing in itself.<br />
<br />
But competition comes with significant costs & downsides:<br />
it's economically 'inefficient', having hundreds of people re-invent the wheel.<br />
In Engineering terms, everyone gets to reinvent the wheel, to make some of the same expensive mistakes<br />
and no one group ever learns all the 'good' things. Until they merge or are bought up...<br />
<br />
Look at what NASA did in the 1960's - from a standing start, they went to the Moon and back, safely.<br />
Now we have multiple private groups trying to get into space, building on a whole different technology base and changing the accepted wisdom and challenging decades of entrenched methods. They lowered the costs by a factor of ten.<br />
<br />
NASA was never challenged and evolved into a bureaucracy, away from its roots: an innovative, daring Engineering organisation. Feynman demonstrated this in the Challenger shuttle explosion investigation.<br />
<br />
Monopolies, or Oligopolies, actively suppress competitors, reduce diversity and multiply costs to consumers.<br />
They may be 'private', but they are anti-competitive and economically disastrous for the markets they hold captive.<br />
Think "Too Big to Fail" and the 2008 global financial crash to understand it's more than just goods affected.<br />
<br />
Open Source actively prevents the worst outcomes of Capitalism (monopolies & 'captured' markets) and encourages the best that Capitalism has to offer - competition of ideas, 'frictionless' sharing of Information and knowledge and 'lifting all boats' or 'standing on the shoulders of giants'.<br />
<br />
Open Source scales up and out, it encourages invention, diversity and novelty and rewards innovation, hard work and risk taking.<br />
Most importantly, it actively prevents asymmetric rewards and 'winner takes all' situations.<br />
It accepts & rewards every small contribution, refinement and step forward, without disadvantaging any competitors.<br />
<br />
This is capitalism in action.<br />
It's mass competition within truly free markets, not stultified & stifled by monopolies playing 'rent-seeker' & extracting a premium via their dominant market position.<br />
<br />
It rewards Great Design and Good Customer Service and 'empowering' your customers.<br />
It's equitable - valuing your business / services based on what you can do, how good you are and solely on your "Value Proposition" to customers, not some artificially constrained & manipulated market-place.<br />
<br />
We've seen this before in the 1960's when Japanese industry knew it might fail and called in W. Edwards Deming. Starting in WWII, Deming had developed and proven a statistically based Quality Improvement model.<br />
Deming's "Quality Circles" transformed Japanese manufacturing, showing up 25 years later when they came to dominate global car manufacturing. Management in the USA thought it was their use of robots, not their embedded Quality Improvement culture which delegated Authority & Responsibility for Quality and Performance to line workers, not 'management'.<br />
<br />
Management also accepted & rewarded feedback and improvements from everywhere, not just from the top of the hierarchy.<br />
<br />
Open Business _is_ risky: you can't slack off, be complacent or stop Putting the Customer First.<br />
You cannot treat you customers poorly or force them to do what's against their interests, but earns you high profits.<br />
<br />
The costs of customers changing providers, or your customers 'substituting' your product with another, are very low.<br />
Open Source lowers the barriers of entry for new players, allowing real Free Markets to operate, encouraging micro-businesses to disrupt large, established players. It doesn't seek to trap customers somewhere, then extract onerous charges 'because we can'.<br />
<br />
A simplistic analysis might brand Open Source "socialist", but it is the precise opposite.<br />
<br />
Open Source underpins fierce, wide-spread competition, while allowing customers to drive innovation with their needs and to benefit from development by all competitors.<br />
Open Source truly levels the playing field, allowing anyone to make a living based on their skills, talent and capability, not an inherited legacy, position or privilege.<br />
<br />
Open Source is an unfamiliar model to many:<br />
it's fierce competition, that embraces sharing, egalitarian market access and equitable rewards for contribution and industry.<br />
<br />
Competitive Open Source is only unappealing to those who think that inequitable rewards are desirable and "winner takes all" produces desirable economic, national and social outcomes. If you want to get rich by exploiting a market position, Open Source won't appeal to you.<br />
<br />
Open Source allows many to share in the Digital and Real Economy, to gain the rewards of the Free Market and to be exposed to the downsides and risks of failing to create and maintain "passionate users".<br />
It's the Epitome of Adam Smith's Free Market, not anything else.<br />
Sharing source code removes information-based market distortions: all sellers and buyers have full market information.<br />
<br />
Google, Facebook and Amazon got huge on the back of Open Source and actively contribute their work back.<br />
Notable, not _all_ their work, but key parts.<br />
Google has given away the Android O/S for the better part of a decade - as a public good to build the Industry, not force people into trade-offs and sub-optimal choices. Microsoft, via key patents, makes more than Google from Android sales.<br />
<br />
These big Internet companies have huge turnovers, have made their shareholders rich and are known for being fiercely competitive. Their success is based on Open Source and they know this, actively supporting developers & projects and contributing back.<br />
<br />
To characterise their commercial success as solely due to Open Source, or within the reach of every 2-person start-up is disingenuous. For every super-large Open Source business, there are tens of thousands of failed startups, thousands of short-lived businesses, hundreds of sustainable mid-scale businesses and a large community of individuals that share and benefit in the work over time.<br />
<br />
Open sharing of Source Code, Data, Documents and Processes is driven in the short-term by profitable commercial activity or altruism funded by research grants or personal interest and exertion.<br />
<br />
In the long-term, we know from Google, Facebook et al (and foundations like 'Apache' and 'GNU'/'FSF'), that money is the fuel for Open Source development - and that is mainly contributed 'in kind' by big corporates acting in their own commercial interests.<br />
<br />
IBM, HP, Redhat & Canonical/ubuntu, even Intel and Microsoft, collectively spend billions each year on software & testing they seemingly 'just' give away. Each of these businesses know precisely their costs and are hard-nosed about every dollar spent. For them, this is not philanthropy, it's a marketing or engineering investment that repays itself many-fold.<br />
<br />
If large Corporations at the heart of the US economy think Open Source makes financial, business and marketing sense, then what is everyone else missing?<br />
<br />
In 2016, we have data on the outcomes of Open Source going back 40 years to the wide-spread sharing of Unix code & utilities.<br />
Whilst the UCB team that delivered the BSD distribution closed down, they continued for as long as they did because of the commercial value realised elsewhere. You can trace the success of the Internet back to the TPC/IP work embodied in "BSD Sockets" and taken up and reused almost everywhere, for free, including Microsoft. That was just one DARPA grant of many.<br />
<br />
BSD's descendants are active and still power commercial enterprises.<br />
<br />
Money makes the world go round and Open Source generates stunningly large sales and profits for those smart enough to embrace it and realise it confers a competitive advantage. It's just good business.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-67684443489499585622014-07-12T14:36:00.000+10:002014-07-12T17:25:35.009+10:00RAID++ and Storage Pools: Leveraging GPT partitions for Asymmetric Media Logical Volumes. Pt 1.This is an exploration of addressing Storage problems posed by directly connected or (ethernet) networked drives, <b>not</b> for SAN-connected managed disks served by Enterprise Storage Arrays.<br />
<br />
<b>The Problem</b><br />
<br />
One of the most important features of the Veritas Logical Volume Manager (LVM) circa 1995 was the ~1MB disk label that contained a full copy of the LVM information of the drive/volume and allowed drives to be renamed by the system of physically shuffled, intentionally or not.<br />
<br />
Today we have a standard, courtesy UEFI, for GUID Partition Tables (GPT) of Storage Devices supported by all major Operating Systems. Can this provide similar, or additional capability?<br />
<a name='more'></a><br />
<br />
LVM didn't just solve problems of providing Data Protection with RAID, Performance Tuning by spreading IO's across spindles and creating larger volumes than were available from single drives, but allowed "live" changes that were previously difficult, error-prone and often time consuming with "stop, reconfigure, rebuild, reboot". Viz:<br />
<br />
<ul>
<li>"just in time" dynamically growing volumes and filesystems, reducing "orphaned" and wasted space from fixed, pre-allocated volumes. Prior to this, it was very easy to run out of filesystem space while across the whole system, more than enough space was available.</li>
</ul>
<ul>
<li>On-the-fly replacement and upgrade of hard disks, by adding new drives as mirrors of physical drives, then removing the old drives from the mirror, once the data was synced.</li>
</ul>
<div>
Computers have migrated out of the back-office, off our desktops and into our pockets to be mobile and constantly with us. They are rapidly becoming both pervasive and invisible. Alongside this, there's been a revolution in Storage, affecting most systems from small (tablets, notebooks, desktops to large servers running Virtual Machines.</div>
<div>
<br /></div>
<div>
Users know that devices are expendable, fungible and replaceable, but <i>data is everything</i>. It's not unreasonable for mobile phone users to demand loss-free Storage and by extension, expect that of every system or service they use. Hard Disks were invented in 1956, the IBM 305 RAMAC, replacing Magnetic Drums as large, fast external Storage and ushering in the current era of Storage.</div>
<div>
<br /></div>
<div>
There are now many choices for Storage Media (PCI-Flash, SSD's, 2.5" HDD's, 3.5" HDD's), different performance & reliability versions of each (Enterprise or NAS Flash and HDD's, fast (10,000RPM) or slow/energy-saving (5400RPM) drives, large 3.5" drives (Shingled Magnet Recording (SMR), 5/6 platter and Helium-filled) or 2.5" drives with 1-, 2-, 3- or 4-platters), and a large choice of connection methods: SD-Card, USB, Thunderbolt/PCIe, SATA/eSATA, SAS and increasingly, ethernet. All of which support removable drives.</div>
<div>
<br /></div>
<div>
A single system may have all or most of these options connected at one time or another, and within even small operations, all media and connection types and can be expected.</div>
<div>
Other media, such as Optical Disk and Enterprise Tape, do exist, but don't commonly support GPT's.</div>
<div>
<br /></div>
<div>
All of which needs to incorporate & transparently interoperate with "Cloud" storage and backup services. Both large Drive Vendors, Seagate and Western Digital, are releasing new drives with native Ethernet interfaces, forcing more changes in the world of Storage.</div>
<div>
<br /></div>
<div>
There are now also many more Storage system operations expected and commonly demanded and used than in 1995:</div>
<div>
<ul>
<li>Snapshots</li>
<ul>
<li>including "differences from a base snapshot", useful in provisioning VM images from a common base.</li>
</ul>
<li>Replication</li>
<ul>
<li>e.g. "rsync"</li>
</ul>
<li>Clones</li>
<li>Versioning and associated Repositories</li>
<li>Migration, especially for VM images between physical hosts.</li>
<li>Continuous Data Protection (CDP)</li>
<ul>
<li>For Disaster Recovery (DR) and Business Continuity, a near current off-site copy of the Storage, accessible by standby servers</li>
</ul>
<li>Backups</li>
<ul>
<li>Full, Incremental, Differential</li>
<li>Bare-metal recovery</li>
</ul>
<li>Archives</li>
</ul>
<div>
It's no longer enough to reply on simple backups or a single Storage Array for an organisations Data Holdings, it's now necessary to catalogue and index all the media and data, preferably in a single place (i.e. Database) that is directly User Accessible. It's expensive and time-consuming forcing Restore requests to be serviced manually.<br />
<br />
"Advanced" features are now demanded at all levels:<br />
<br />
<ul>
<li>Compression</li>
<li>Long-range Compression & DeDupe (<i>lrzip</i>)</li>
<li>De-Duplication</li>
<li>Encryption, public-key based</li>
</ul>
</div>
</div>
steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-36205141216188845472014-07-10T13:31:00.002+10:002014-07-10T13:31:40.853+10:00RAID++ and Storage Pages: We may be asking the wrong questionsThe implied contract between Storage Devices, once HDD's only, and systems is a rather weak one.<br />
<blockquote class="tr_bq">
Storage Devices return blocks of data on a "Best Efforts" basis, failure & error handling are minimalist or non-existent.</blockquote>
There's no implicit contract with the many other components that are now needed to move data off the Storage Device and into Memory, HBA's, cables, adaptors, switches etc. The move to Ethernet and larger Networks compounds the problem: networks are not nearly error-free. This matters when routinely moving around Exabytes and more: errors and failures are guaranteed for any human-scale observation period.<br />
<br />
Turning this weak assurance into usable levels of Reliability and Data Durability is currently left to a rather complex set of layers, which can have subtle & undetectable failure modes or in "Recovery" mode, have unusably poor performance and limited or no resilient against additional failures. We need to improve our models to move past current RAID schemes to routinely support thousands of small drives and new Storage Class Memory.<br />
<br />
Scaling Storage to Petabyte and Exabyte sized Pools of mixed technologies needs some new thinking.<br />
New mixed technologies now provide us with multiple Price-Size-Performance components, requiring very careful analysis to optimise Systems against owner criteria.<br />
<br />
There is no one true balance between DRAM, PCI-Flash, SSD's, fast-HDD, slow-HDD and near-line/off-line HDD or tape and Optical Disk. What there <i>is</i>, is a willingness of an owner to pay. Presumably they have a preference to pay <i>enough</i>, but not significantly more, for their desired or required "performance", either as "response time" latency or "throughput". Very few clients can afford, or need, to store everything in DRAM with some sort of backup system. It's the highest performance <i>and</i> highest priced solution possible, but is only necessary or desirable in very constrained problems.<br />
<br />
DRAM is around $10/GB, Flash and SSD about $1/GB and HDD's from $0.04 to $0.30/GB for raw disk.<br />
<br />
Here's a possible new contract between Storage Devices and Clients/Systems:<br />
<blockquote class="tr_bq">
Data is returned Correct, Complete and Verifiable, in whole or part, between the two Endpoints.<a name='more'></a></blockquote>
To explain and unpack that.<br />
<br />
<b>Between Endpoints, or End-End Checks.</b><br />
<br />
Systems use many logical abstractions, like Filesystems and Virtual Memory, that don't just make it easy for Programmers, but allow systems designers to improve Performance (e.g. caching) and to transparently detect and correct Errors. This is only possible with very strong models, especially when formally analysed, and "orthogonal" concerns can be separated outsteve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-76066138049131716882014-06-22T14:57:00.002+10:002014-06-22T14:57:14.618+10:00RAID++: So, you cant afford the extra cost of Data Protection at $0.10-$0.20 per GB?<b>Summary</b>: You and your business probably now depend on computers and smartphones/tablets for most of your daily work and other activities. If you don't pay up-front to protect your data, you'll pay for it many times over at a later date, <i>when</i>, not if, you have a drive fail and lose all data.<br />
<br />
When data is $0.20/GB (or even $1/GB) and wages are $35-60/hour and it will take a minimum of 1 day to reconstruct data, more likely a week+, spending a little money up-front for Data Protection seems prudent to me.<br />
<br />
The <a href="http://www.eecs.berkeley.edu/Pubs/TechRpts/1987/CSD-87-391.pdf">1987 Berkeley RAID paper</a> was written at a time few people had PC's and storage cost $40,000/GB in current dollars. The economics of swapping space for computation were compelling at the time, nowdays, very few people have even$1,000 invested in Disk Storage, let alone $250,000.<br />
<br />
Good desktops or laptops are now available in $500-$1,000 range, with <a href="http://stevej-lab-notes.blogspot.com.au/2014/05/retail-disk-prices-sorted.html">Commodity Drives</a> costing $0.04-$0.10/GB and <a href="http://stevej-lab-notes.blogspot.com.au/2014/06/retail-disk-prices-enterprise-drives.html">Enterprise Drives</a> from $0.12-$0.65/GB, and more for high-spec variants. Times are very different: raw prices have fallen 500,000, Bit Error Rates (BER/UBER) are up ~100 times, Mean Time Between Failures (MTBF) have increased 10-100 fold, raw read/write rates have increased 100-300 times, while access times (rotation & seek) are 2-5 times different. As is estimated disk utilisation: at some point after 2000 the average drive went for 90%-100% full to ~75%, at least for Desktops. This suggests that drives are now "Big Enough" and not a System Constraint, at least not for Capacity. The advent of affordable, large Flash Memory with reasonable read/write speeds and uniform access times has removed one of the big constraints of storage: random I/O per second.<br />
<br />
Researching RAID designs, I was surprised by I.T. Professionals and home users alike, that baulk at the cost of reasonable Data Protection, even $100 for a single USB drive plus a 4-drive NAS unit is definitely "too expensive" (<$1,500). Do they have such volumes of data that the cost of extra drives is overwhelming? Or is the data worth so little, or cost so little, that it's not worth protecting?<br />
<a name='more'></a>Data likely to be stored on home and micro-business drives is likely to be of three types, each with a different cost for Data Loss events:<br />
<ul>
<li>Owner created content</li>
<ul>
<li>What's would it cost to recreate the data?</li>
<ul>
<li>For business records, recapturing client data can be difficult.</li>
</ul>
<li>Is there an Opportunity cost if the data is lost?</li>
<li>Are there attributable use or sentimental costs for lost data or records?</li>
</ul>
</ul>
<ul>
<li>Licensed content</li>
<ul>
<li>It may not still be available.</li>
<li>There may be high additional license fees to replace from the owner/vendors.</li>
</ul>
</ul>
<ul>
<li>Free (downloaded) content</li>
<ul>
<li>May no longer be available on-line.</li>
</ul>
</ul>
As well, there is a real, attributable cost to downloading files in Australia:<br />
<ul>
<li>the cost per GB in the ISP plan, especially via Mobiles with punitive excess data charges, and</li>
<li>the time cost of locating then downloading new copies of files.</li>
</ul>
The easiest and fastest means to good Data Protection for low-end applications is to use one of the many Internet storage services. This option isn't available to most homes and micro-businesseses.<br />
<br />
In Australia with its poor Broadband options, uploading speeds are constrained to 0.5-1Mpbs with ADSL2, approx 100-400MB/hour, depending on ISP congestion and Network Contention Ratios dimensioned in. Uploading or backing up just 1GB could take 10 hours, while 1TB, a small drive, will take over a year, if the link stays up and the error rate isn't too high. Until guaranteed, low-contention upload over 40Mbps is available in Australia, on-line backups, even "differential", upload/backups using Internet Storage Services are not viable. This is different to storing pictures from your smartphone. It's a comprehensive, structured and continuing "second copy" of your precious data, with searchable indexes and per-copy contents list. A full copy of your data isn't useful if you can't locate, or name, files and folders you need back.<br />
<br />
Whilst Internet Storage Services may offer lots of space and reasonable levels of data protection, they have two "drop-dead" problems:<br />
<ul>
<li>sudden and complete Data Loss due to non-technical problems. If they go bust, one day all your Data is gone for good.</li>
<ul>
<li>Data that you no longer own and becomes part of the assets of the distressed business, to be sold to the highest bidder for any purposes they desire.</li>
<li>No vendor I've seen offers a dedicated, clear-title asset for drives, but I haven't looked hard. The closest might be the many "dedicated mac mini hosting" services.</li>
</ul>
</ul>
<ul>
<li>Even if you have warning of impending collapse, the "rush on the bank" effect means <i>nobody</i> will be able to retrieve more than a few hundred MB of data, if anything.</li>
<ul>
<li>Like a bank, if there's a whiff of trouble, everyone will want their money, or data, back, <i>right now</i>!</li>
<li>Business will only be in trouble if they're pushing the financial envelope, so they'll minimise expenses, including "just enough" bandwidth and small per-user quotas.</li>
<li>These two factors, correlated infinite demand and thin pipes, constitute a very effective Distributed Denial of Service attack (DDoS). As everyone rushes to get their data, they block access to the very thing they want. If the hosting service hasn't taken steps to limit congestion, allowing unlimited connections, then the link will be permanently saturated, but no useful traffic will flow. All connections will time-out and restart.</li>
</ul>
</ul>
For Australians, and probably many people overseas, they need to fully curate and maintain <i>their own data holdings</i>. If your business, or just personal tax/income records, are on a computer, you need a minimum of three independent copies of the data, at least one of which needs to be stored at least 16km (10 miles) away in a secure, clean, dry, cool location.<br />
<br />
Running just external drives and a software equivalent to Apple's "Time Machine", then if you have 1TB of data, a portable USB drive will cost ~A$75-$100. You need three and you need to replace them - drives have a 5 year design-life. At best, that's $250 every 5 years, or $50/year, around 1hour 20 minutes of your work rate,<i> 1min 30seconds per week</i>, just for the drives. You then need to cost your time in swapping around those drives and transporting<br />
<br />
Backups are <i>only</i> useful if you can restore from them. Is any home user or micro-business really going to spend time every 6-12 months confirming all drives are readable and without error? That's at least a 2-4 hour task. At any hourly rate, it gets expensive, quickly.<br />
<br />
If you think "I can just use Flash Drives or DVD's", think again. You'll still need at least two copies of your data on top of your main copy, and one must be off-site and well maintained. At $1/Gb for Flash (times two), you're up for $2,000 for 1TB, or storing just 50GB, one or two hours of video. DVD's, even at $0.25/disk are the same price as Flash: $1/GB. Both Flash and writeable DVD's "fade" over time. Flash chips wear out (fail to store data) and lose charge, while the dyes in DVD's fade. It makes sense to write all you accounts/income data to a set of Flash drives that you rotate every day and to regularly replace all those drives.<br />
<br />
That's a <i>per machine</i> cost and doesn't include smartphones, tablets and music or video collections.<br />
Most households now have multiple devices, per person, and run an ethernet network, either wireless, hard-wired or both. Low-end NAS, or Network Attached Storage, devices that connect to home networks come out at around $100/drive-bay <i>plus drives</i> (~$250 for 2-bays and $400 for 4-bays).<br />
<br />
Many NAS appliances also have a USB port to attach additional drives. I've no idea if they can be used to create copies for off-site storage.<br />
<br />
Using 3.5" drives at $150-$250 each (commodity or NAS-grade) gives you a second copy of your data, shared across all machines, for $500-$1000. With a 5-year drive life and 3% Annualised Failure Rate (20% extra for failed drives over 5 years), it's $125 to $250/year: or 4-8 hours of wages per year (2-4 hours for a business).<br />
<br />
That's the trade-off: unknown time spent recreating your records and files after losing a disk, always at an inconvenient time & not one of your choosing, or trade a half-day of wages each year against that eventuality.<br />
<br />
If you have a hobby where you store and use large amounts of data, say Photography or Video, the chances are, you're already using an internal or directly connected RAID device, just to store everything and to get reasonable speed. You've most likely already experienced a Data Loss Event and will have developed a backup regime to suit your needs and budget.<br />
<br />
<hr />
<br />
Wage Rate Calculations<br />
<br />
average full-time wage: $75,000/year<br />
<b>naive rate</b>: $35.93/hour [40 hour week, 52 weeks/year]<br />
<br />
take-home: $56,250/year<br />
<b>take-home hourly rate</b>: $34.09/hour [37.5 hour week + 20 days vacation, 10 days public holidays, 5 days sick leave]<
cost to business salary: $118,177.50/year [30% extra "on-costs"]
per-position yearly salary: $153,630.75/year [100% coverage of a position for 50 hours/week service]
<b>on-cost hourly rate to business: </b>$59.09/hour [37.5 hour week etc]<br />
<br />
37.5 hour week: 1650 hours/year [4 weeks vacation, 10 public holidays, 1 week sick leave]<br />
52 weeks @ 40-hours: 2087 hours/year<br />
52 weeks @ 10hr-shifts, 5 days/week: 2600 hours/year<br />
multiplier: 1.5757 [from 37.5 hours to 10-hour/day 5days/week coverage]steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-68075360784816283422014-06-20T16:24:00.002+10:002014-06-20T16:24:24.131+10:00Recruiting FAIL: Update with ITCRA documentsI've put on-line the follow-up documents [links below fold] I received from ITCRA on my complaint, lodged in Aug/Sep 2012 and apparently resolved before Christmas that year. I have on record, because almost all my interactions have been via email, that I wasn't informed on the lack of Natural Justice until around six months after I lodged my grievance.<br />
<br />
I was given an undertaking that a) ITCRA would write a Case Study from my complaint, in lieu of informing me of their determinations and actions taken, and b) I would be sent a copy.<br />
<br />
A year on, Dec 2013, I enquired after the Case Study. My guess from the delay is that it'd never been written, as promised in writing.<br />
<br />
It was only in February this year, 2014, that I received that Case Study.<br />
In a separate email I was informed that the Agent had been dismissed over this matter, presumably in late December 2012.<br />
<a name='more'></a><br />
<br />
Links, PDF documents, downloadable:<br />
<a href="https://drive.google.com/file/d/0B1GLdfqdwpNQRmZEZEVyckdBcHM/view">ITCRA Complaint transmitted to Agency. Undated.</a><br />
<a href="https://drive.google.com/file/d/0B1GLdfqdwpNQenlWY3lyZnNtQmM/view">First notice to Agency of grievance sent to Agency. Undated.</a> [I've incorrectly annotated with a 2011 date.]<br />
<a href="https://drive.google.com/file/d/0B1GLdfqdwpNQcU5IUnBHODZqMEE/view">ITCRA email to me advising Agent was fired over this.</a> 12-Feb-2014.<br />
<a href="https://drive.google.com/file/d/0B1GLdfqdwpNQTDdYenRIMjlaYjA/view">ITCRA Case Study. Undated,</a> received 12-Feb-2014.<br />
<br />
Previous posts:<br />
<a href="http://stevej-on-it.blogspot.com.au/2012/12/recruiting-fail-non-response-of-itcra.html">12-Dec-2012.</a> on ITCRA'a non-response.<br />
<a href="http://stevej-on-it.blogspot.com.au/2012/09/recruiting-fail-part-3-itcra-complaint.html">Part 3 of the series</a> on lodging a complaint.<br />
<a href="http://stevej-on-it.blogspot.com.au/2012/09/recruiting-fail-how-to-foul-up-employee.html">Part 1 of the series,</a> detailing events.<br />
<br />
But on what grounds was he dismissed? These were never made clear to me.<br />
<br />
This is important as both my Complaint and the formal Complaint transmitted to the Agency included <i>multiple</i> breaches of the Australian Consumer Law (ACL, previously TPA), which were cause for action by me against the Agent and the Agency.<br />
<br />
That both ITCRA and hence the Agency were aware of these prima facé breaches of the ACL are spelled out clearly and unequivocally in this these words from the Case Study :<br />
<blockquote class="tr_bq">
<b>The Question</b>They (the Agency) ask whether they can be held liable for anything. </blockquote>
<blockquote class="tr_bq">
<b>The Answer</b>Yes.</blockquote>
Does deliberately not informing me of their Determination, actions required against the Agent and by the Agency and withholding relevant information for a very long period amount, in itself, to a breach of the ACL, specifically unconscionable conduct by ITCRA and its various employees?<br />
<br />
Were the ITRCA Board, Disciplinary Committee and CEO all obliged under the ACL, as to withhold pertinent information would be "misleading and deceptive", even unconscionable conduct, to inform me that their consideration of the facts, and backed by their action against a member and employee, was sufficient evidence for me to take action under the ACL for real damages, viz lost wages and damage to my reputation amongst all ITCRA members who became aware of my complaint?<br />
<br />
The very determination against the Agent/Agency, based on the headline complaint, and its acceptance by them, would seem to be an unequivocal admission of guilt and responsibility of breaches of the ACL.<br />
<br />
It took a very long time for ITCRA to recruit a Disciplinary Committee for this matter as so many Agencies had my CV on their books, either then or at some time. To ascertain this, all those Agencies had to be contacted and the reason given. The only question is how much detail of my complaint was passed around ITCRA members.<br />
<br />
Does the denial of Natural Justice in the ITCRA 'complaints' process for members of the <i>public</i> against ITCRA members amount to a breach of any Act and grounds for action?<br />
<br />
Does the extremely tardy and reluctant action of ITCRA and its officers, after <i>offering</i> undertakings to provide me with information about my complaint via an anonymised Case Study, amount to "misleading and deceptive" conduct under the ACL? Specific promises weren't just entered into, but volunteered by them.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com1tag:blogger.com,1999:blog-29583699.post-89758636802217292392014-06-19T20:18:00.000+10:002014-06-20T13:56:12.266+10:00Recruiting FAIL: The Gap between Promise and Reality<a href="http://stevej-on-it.blogspot.com.au/2012/12/recruiting-fail-non-response-of-itcra.html">Following from my own problems with recruiters</a>, a mate sent me the following unsatisfying exchange with a "recruiting" agent, from a supposedly specialist firm.<br />
<br />
If you're looking for work, you'll be only too familiar with this mechanical approach where they don't even both to look at your CV and confirm there's a hint of a match.<br />
<br />
If you're hiring, this is death. The High Priced Agency might send you warm bodies, but they've done less than you'd want, and possibly would like to believe.<br />
<br />
This led me to spending some time researching these "experts".<br />
<br />
It's a public company in the UK that sells franchises all around the world - they claim 67 locations and 2,500 staff.<br />
To investors, they claim "high value, high margin". They make a <i>motza</i> out of the suckers on both sides of the table - just look at their financials.<br />
<br />
Rummaging around in the company and business names registry and you don't find nearly the entities widely claimed on-line. Is that a problem under the Trade Practice Act (1974) now ACL (2010) part of the CCA (2010) or could someone just register those names and sue them for using them? This is why<a href="http://en.wikipedia.org/wiki/Hungry_Jack's"> we have "Hungry Jacks" not "Burger King" burgers</a>. Someone didn't register the names across Australia.<br />
<br />
Dig even a little and you find stuff you'd rather not know. A formal warning from ACMA over spamming, a Fairwork complaint on unfair dismissal which doesn't just question the judgement of managers, but makes a case for the employee being consistently 'loose with the truth'.<br />
<br />
Then there's the reviews by clients... A few "5-star" ratings, some which read like they were purchased from a sweat-shop in India, others that jibe totally with the rest.<br />
<br />
Oh, and one of the original principals who's now worth tens of millions of dollars after the 2005 float is best described as "colourful", four wives, lots of bling and a sting operation for cocaine use.<br />
<br />
<hr />
<blockquote class="tr_bq">
Date: 19 June 2014<br />
Subject: Re: UX Designer contract role<br />
To: recrutier@progressiverecruitment.com</blockquote>
<blockquote class="tr_bq">
Dear XXXX,</blockquote>
<blockquote class="tr_bq">
With respect, the problem with most I.T. recruiting agencies is just this - a data mining program trolled the 1000's of CV's that you have online and sent me this "opportunity". If you took the time to read my resume, perhaps schedule a face to face interview, get to know me, my skills, my strengths and weaknesses, shared a cup of tea, a Skype meeting, maybe then you would be motivated to get me something that I am more suited too.</blockquote>
<blockquote class="tr_bq">
This is the problem - the gap between the many people with the skills and the actual work has been filled with huge "talent agencies" and programs developed by the likes of myself. Programs that perhaps don't reflect a person's skills or character.</blockquote>
<blockquote class="tr_bq">
I have HP-UX on my resume, NOT UX. Perhaps I could do this job with proper coaching, but would you actually take the time to read my resume and find out? Give me some Skype time?</blockquote>
<blockquote class="tr_bq">
[snip]</blockquote>
<blockquote class="tr_bq">
I realize costs have been cut world wide in a lot of industries but this whole process adds a layer of complexity that I find fraustrating. How many times a day do I need to update my resume with "keywords"? before I actually get an interview with you, then maybe the other two people and then the test?</blockquote>
<blockquote class="tr_bq">
I've been with your agency for over 2 years, maybe longer. I have applied for:</blockquote>
<ul>
<li>Unix / Linux System Administration (the buzz word is "Engineer" or "Designer" these days)</li>
<li>Windows Server Administration 2003, 2008, 2012 - one year relevant experience</li>
<li>CISCO CCNA 2 Network Admin Certificate - with 2 1/2 years relevant experience managing a LAN / WAN and its hardware (modems, routers, cabling, switches, etc)</li>
<li>Experience with all workstations - 15 years of experience building, troubleshooting, designing, managing.</li>
<li>AMD and Intel architecture hardware - 4 years of in depth knowledge that makes me as good as at least a 1st year apprentice electronics technician</li>
<li>iphones, android phones - very good knowledge</li>
<li>Virtual machines - very good - set up at least 15</li>
<li>Scripting in Perl and Shell - very good</li>
<li>Programming in Visual Basic and C++ - good</li>
<li>Database scripting (MySql) - good</li>
</ul>
<blockquote class="tr_bq">
[snip]</blockquote>
<blockquote class="tr_bq">
Sincerely,</blockquote>
<blockquote class="tr_bq">
BP</blockquote>
<hr />
<blockquote class="tr_bq">
Hi B, </blockquote>
<blockquote class="tr_bq">
I Hope you have been well. </blockquote>
<blockquote class="tr_bq">
Are you looking for an exciting 3-6 month contract opportunity in the UX design space? </blockquote>
<blockquote class="tr_bq">
My client is looking for a UX Designer who will need to work on an old product that needs re-vamping. As the sole UX Designer you will design, lead, and analyze end user needs and leverage the findings into site architectures, Wire-frames, and functional specification documentation for web sites and applications. The UX designer will collaborate closely with a product manager and developers to create the best possible user experience. </blockquote>
<blockquote class="tr_bq">
[snip]</blockquote>
<blockquote class="tr_bq">
If this sounds like you please register your immediate interest by sending through your updated CV and a link to your portfolio/creative work. </blockquote>
<blockquote class="tr_bq">
Kind regards,<br />
XXXX</blockquote>
<blockquote class="tr_bq">
Progressive ICT</blockquote>
<hr />
Business Names Search<br />
<a href="https://connectonline.asic.gov.au/RegistrySearch/faces/landing/bn/SearchBnRegisters.jspx?_adf.ctrl-state=ig8ji303r_13">https://connectonline.asic.gov.au/RegistrySearch/faces/landing/bn/SearchBnRegisters.jspx?_adf.ctrl-state=ig8ji303r_13</a><br />
<br />
ORGTEL, TAS BN01642042, Registration Date 01/10/2009<br />
<a href="http://www.search.asic.gov.au/cgi-bin/gns030c?acn=139699148&juris=9&hdtext=&srchsrc=1">http://www.search.asic.gov.au/cgi-bin/gns030c?acn=139699148&juris=9&hdtext=&srchsrc=1</a><br />
<br />
Name STHREE AUSTRALIA PTY LIMITED<br />
ACN 126 409 103<br />
ABN 86 126 409 103<br />
Registration Date 06/07/2007<br />
<a href="http://www.search.asic.gov.au/cgi-bin/gns030c?acn=126409103&juris=9&hdtext=ACN&srchsrc=1">http://www.search.asic.gov.au/cgi-bin/gns030c?acn=126409103&juris=9&hdtext=ACN&srchsrc=1</a><br />
<br />
Current details for ABN: 86 126 409 103, ACN: 126 409 103<br />
STHREE AUSTRALIA PTY LIMITED, from 1-Dec-2007<br />
HUXLEY ASSOCIATES, Business Name<br />
<a href="http://abn.business.gov.au/SearchByAbn.aspx?StartSearch=True&SearchText=126409103">http://abn.business.gov.au/SearchByAbn.aspx?StartSearch=True&SearchText=126409103</a><br />
<br />
Name: STHREE AUSTRALIA PTY LIMITED<br />
ACN: 126 409 103<br />
ABN: 86 126 409 103<br />
Registration date: 6/07/2007<br />
Next review date: 6/07/2014<br />
<span style="background-color: lime;">Former name(s): PROGRESSIVE RECRUITMENT PTY LIMITED</span><br />
<br />
NOTE: There are <b>no</b> current entity, trading or business name registered in Australia besides "SThree Australia" and "Huxley Associates". Claims that the other entities are "Trading Divisions" within Australia can only be internal accounting arrangements, not backed by any formal entity or registration.<br />
<br />
Only in Tasmania is there a registered trading name: "Orgtel."<br />
<br />
<hr />
Progressive GE (Global Energy)<br />
<a href="http://www.progressivege.com/about-us">http://www.progressivege.com/about-us</a><br />
<a href="http://www.progressivege.com/company-details">http://www.progressivege.com/company-details</a><br />
<br />
Progressive Recruitment<br />
<a href="http://www.progressiverecruitment.com/about-us">http://www.progressiverecruitment.com/about-us</a><br />
<a href="http://www.progressiverecruitment.com/company-details">http://www.progressiverecruitment.com/company-details</a><br />
<br />
Huxley Associates<br />
<a href="http://www.huxley.com/company-details">http://www.huxley.com/company-details</a><br />
<br />
Real Staffing<br />
<a href="http://www.realstaffing.com/company-details">http://www.realstaffing.com/company-details</a><br />
<br />
<br />
SThree info.<br />
<a href="http://www.sthree.com/about-us/our-philosophy">http://www.sthree.com/about-us/our-philosophy</a><br />
<br />
SThree on Wikipedia<br />
<a href="http://en.wikipedia.org/wiki/SThree">http://en.wikipedia.org/wiki/SThree</a><br />
<br />
SThree History via Archive.org. Started 1986 as "Computer Futures", Taken Public in 2005.<br />
<a href="https://web.archive.org/web/20130207013722/http://www.sthree.com/en/page/history/">https://web.archive.org/web/20130207013722/http://www.sthree.com/en/page/history/</a><br />
<br />
SThree Investment Case: High Value, High Margin<br />
<a href="https://web.archive.org/web/20130207013827/http://www.sthree.com/en/page/investment_case/">https://web.archive.org/web/20130207013827/http://www.sthree.com/en/page/investment_case/</a><br />
<br />
Live site<br />
<a href="http://www.sthree.com/investors/investment-case">http://www.sthree.com/investors/investment-case</a><br />
<br />
SThree global businesses<br />
<a href="http://assets.sthree.com/legal/SThree_Group_Companies.htm">http://assets.sthree.com/legal/SThree_Group_Companies.htm</a><br />
<br />
<hr />
Gary Goldsmith, one of original owners of "Computer Futures". Described as "Colourful.<br />
<a href="http://www.express.co.uk/news/royal/378060/Duchess-of-Cambridge-s-very-colourful-uncle">http://www.express.co.uk/news/royal/378060/Duchess-of-Cambridge-s-very-colourful-uncle</a><br />
<br />
Treffry v SThree Australia Pty Ltd [2013] FWC 3697.<br />
Odd behaviour by both employee and company is specifically commented upon.<br />
<a href="http://www.vta.vic.edu.au/docs/News/HR%20Update%20August%202013.pdf">http://www.vta.vic.edu.au/docs/News/HR%20Update%20August%202013.pdf</a><br />
<blockquote class="tr_bq">
Senior Deputy President (SDP) Boulton found that there was a valid reason for terminaton statng that “there is evidence of dishonesty which was premeditated, preconceived, and which was perpetuated, even after it had been challenged by SThree and that Mr Treffry was warned about possible disciplinary acton”. Further to that, he noted that given the relatvely short period of employment, being about one year and three months, the fact that the conduct of Mr Treffry was clearly dishonest, it would have the potental to undermine the trust and confidence which is necessary in the employment relatonship. SDP Boulton did have some reservatons as to whether the terminaton might be correctly characterised as being one for serious misconduct. “In any event, I am not satsfied that the characterisaton of the terminaton as being for serious misconduct would be such as to outweigh all the other consideratons which have led to the conclusion that the terminaton of Mr Treffry’s employment was not harsh, unjust or unreasonable”.</blockquote>
<br />
ACMA, 2012. Formal Warning under Spam Act issued to SThree Australia<br />
<a href="http://152.91.62.26/Industry/Marketers/Anti-Spam/Ensuring-you-dont-spam/sydney-businesses-failed-to-comply-with-spam-act-i-acma">http://152.91.62.26/Industry/Marketers/Anti-Spam/Ensuring-you-dont-spam/sydney-businesses-failed-to-comply-with-spam-act-i-acma</a><br />
<a href="http://www.acma.gov.au/webwr/_assets/main/lib410040/sthree_australia-s41_formal_warning-6jan2012.pdf">http://www.acma.gov.au/webwr/_assets/main/lib410040/sthree_australia-s41_formal_warning-6jan2012.pdf</a><br />
<br />
<br />
Progressive Recruitment Specialists <i>Reviews</i> - www.progressive.co.uk <br />
<a href="http://www.reviewcentre.com/reviews132341.html">http://www.reviewcentre.com/reviews132341.html</a><br />
<a href="http://www.reviewcentre.com/r132341_2_Progressive_Recruitment_Specia.html">http://www.reviewcentre.com/r132341_2_Progressive_Recruitment_Specia.html</a><br />
<br />
<br />
<hr />
Business Names search - '126409103'<br />
Business Names search - '126 409 103'<br />
Business Names search - 'STHREE AUSTRALIA PTY LIMITED'<br />
Organisations & Business Names search - 'STHREE AUSTRALIA PTY LIMITED'<br />
<br />
<br />
<hr />
Business name: <b>ORGTEL</b><br />
Status: <span style="background-color: lime;">Registered</span><br />
Registration date: 1/10/2009<br />
Renewal date: 1/10/2015<br />
Address for service of documents: Level 9 1 Market St Sydney NSW 2000<br />
Principal place of business: Level 9 1 Market St Sydney NSW 2000<br />
Holder Name: STHREE AUSTRALIA PTY LIMITED<br />
Holder Type: Body Corporate<br />
ABN: 86 126 409 103<br />
<br />
Former identifier: BN01642042<br />
Former State/Territory: <span style="background-color: lime;">TAS</span><br />
<br />
<br />
<hr />
Business name: PROGRESSIVE RECRUITMENT<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 21/01/2008<br />
Renewal date: 21/01/2011<br />
Cancelled date: 21/01/2011<br />
<br />
Former identifier: B2068812U<br />
Former State/Territory: VIC<br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 10/11/2009<br />
Renewal date: 10/11/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN21413800<br />
Former State/Territory: <span style="background-color: lime;">QLD</span><br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 26/10/2009<br />
Renewal date: 26/10/2012<br />
Cancelled date: 11/03/2011<br />
<br />
Former identifier: B2229987W<br />
Former State/Territory: <span style="background-color: lime;">VIC</span><br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 4/11/2009<br />
Renewal date: 4/11/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN01646217<br />
Former State/Territory: <span style="background-color: lime;">TAS</span><br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 27/10/2009<br />
Renewal date: 27/10/2012<br />
Cancelled date: 10/03/2011<br />
<br />
Former identifier: BN11447059<br />
Former State/Territory: <span style="background-color: lime;">WA</span><br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 5/11/2009<br />
Renewal date: 5/11/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN05058880<br />
Former State/Territory: <span style="background-color: lime;">SA</span><br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 28/10/2009<br />
Renewal date: 28/10/2012<br />
Cancelled date: 24/03/2011<br />
<br />
Former identifier: F00136100<br />
Former State/Territory: <span style="background-color: lime;">ACT</span><br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 27/10/2009<br />
Renewal date: 27/10/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN01134048<br />
Former State/Territory: <span style="background-color: lime;">NT</span><br />
<br />
<br />
<hr />
Business name: COMPUTER FUTURES<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 26/10/2009<br />
Renewal date: 26/10/2012<br />
Cancelled date: 9/03/2011<br />
<br />
Former identifier: BN98463286<br />
Former State/Territory: <span style="background-color: lime;">NSW</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 4/11/2009<br />
Renewal date: 4/11/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN01646229<br />
Former State/Territory: <span style="background-color: lime;">TAS</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 26/10/2009<br />
Renewal date: 26/10/2012<br />
Cancelled date: 11/03/2011<br />
<br />
Former identifier: B2229974L<br />
Former State/Territory: <span style="background-color: lime;">VIC</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 27/10/2009<br />
Renewal date: 27/10/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN01134050<br />
Former State/Territory: <span style="background-color: lime;">NT</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 26/10/2009<br />
Renewal date: 26/10/2012<br />
Cancelled date: 9/03/2011<br />
<br />
Former identifier: BN98463284<br />
Former State/Territory: <span style="background-color: lime;">NSW</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 28/10/2009<br />
Renewal date: 28/10/2012<br />
Cancelled date: 24/03/2011<br />
<br />
Former identifier: F00136099<br />
Former State/Territory: <span style="background-color: lime;">ACT</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 27/11/2009<br />
Renewal date: 27/11/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN21436927<br />
Former State/Territory: <span style="background-color: lime;">QLD</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 5/11/2009<br />
Renewal date: 5/11/2012<br />
Cancelled date: 10/03/2011<br />
<br />
Former identifier: BN11453424<br />
Former State/Territory: <span style="background-color: lime;">WA</span><br />
<br />
<br />
<hr />
Business name: JP GRAY<br />
Status: <span style="background-color: lime;">Cancelled</span><br />
Registration date: 5/11/2009<br />
Renewal date: 5/11/2012<br />
Cancelled date: 18/02/2011<br />
<br />
Former identifier: BN05058892<br />
Former State/Territory: <span style="background-color: lime;">SA</span>steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-31675856854545620652014-06-15T16:25:00.000+10:002014-06-19T16:59:30.498+10:00RAID++: Erasures aren't ErrorsI'm following the Communications / Transmission theory nomenclature of "Erasures" to mean drive failure ('no signal') and "Errors" to mean incorrectly sent & received symbols.<br />
<br />
With 5,000 x 2.5" 2TB drives in a single rack, what hardware problems will we experience?<br />
Worldwide, it's assumed a single vendor might sell 100,000 of these arrays.<br />
<ul>
<li>How many read errors should we expect? "<i>Errors"</i></li>
<li>How many drives will fail in a year? "<i>Erasures"</i></li>
<ul>
<li><i>I don't have numbers for other parts like fans, PSU's, boards, connectors, RAM.</i></li>
<li>How many RAID rebuilds?</li>
<li>How long will they take?</li>
<li>How many dual-failures might we expect?</li>
</ul>
</ul>
We can use RAID-1 to set the minimum baseline {cost, performance, data-protection/data-loss} to compare other schemes against. <a href="http://stevej-lab-notes.blogspot.com/2014/06/raid-1-errors-and-erasures-calculations.html">Calculations, here</a>.<br />
<a name='more'></a><br />
<b>Specs</b><br />
<br />
I'm going to assume 100% power-on time (24/7 operations) and a 64% Duty Cycle, excluding RAID activity (rebuilds and data scrubbing).<br />
With average 64KB transfers, this gives throughput of 20TB/drive/year of user data. Only hi-spec drives are designed for this duty cycle and 24/7 usage.<br />
<br />
AFR, Annualised Failure Rates, the inverse of MTBF, is spec'd at 0.8%-1.2%, while real-world reports suggest 3% AFR's (250,000hr MTBF), used here.<br />
<br />
Drive design life is thought to be 5 years, accounting for both wear&tear and effects like lubricant evaporation. Vendors cannot control the use or environment of their products.<br />
<br />
Drive Bit Error Rate (BER or UBER for Unrecoverable BER) is quoted at 10<sup>-14</sup> for consumer-grade drives and 10<sup>-16</sup> for hi-spec drives. Exactly what this means with 4KB (32kbit) blocks is unclear: drive vendors don't publish on this topic. This piece will use the raw figure and assume only single-bit errors within a block, though the literature suggests media defects cause long error bursts, just as scratches and pinholes do on Optical disks.<br />
<br />
For around 20 years, since drives moved to LBA (Logical Block Addressing), they've silently handled "bad blocks" or 'hard' media errors. Data is probably collected by S.M.A.R.T. monitoring, but I've no good studies to guide me. One of the large 2009 studies (Google?) said there was little correlation between S.M.A.R.T. errors and sudden drive failure. Firmware on consumer drives does automatic retries (6-8 times) in the face of errors, resulting in occasional long response times, as each retry loses a full drive revolution (8.4-11.1msec). Storage Arrays disable this automatic drive retry/recovery because they have higher-level error recovery. However, drives still need to handle 'soft' errors due to tracking errors induced by outside factors, such as the noise and vibration found in high-density Arrays. This piece ignores 'soft' errors and bad-block handling.<br />
<br />
There are two main drive speeds available: 5,400RPM and 7,200RPM, with 10,000RPM available at a power & cooling, price and maybe reliability premium. The rough throughput figures are below, with average seek time guessed at 3msec. With ~1Gbps sustained transfer rates, a 4KB block (32K bits) would take 0.032msec to read/write, below our precision, while a 64KB transfer (0.5Mbit) takes 0.5msec, affecting IO/sec.<br />
<br />
<table frame="box"><thead>
<tr><th>RPM</th><th>Avg latency</th><th>0ms seek<br />
IO/sec</th><th>3ms seek<br />
IO/sec</th><th>3ms seek<br />
+0.5ms trf<br />
IO/sec</th></tr>
</thead><tbody>
<tr><td colspan="5"><hr />
</td></tr>
<tr><td>5,400</td><td>5.6msec</td><td>180</td><td>116.88</td><td>110.43</td></tr>
<tr><td>7,200</td><td>4.2msec</td><td>240</td><td>139.54</td><td>130.44</td></tr>
<tr><td>10,000</td><td>3.0msec</td><td>333</td><td>166.67</td><td>153.85</td></tr>
</tbody><caption align="bottom"><b>Drive throughput</b><b> vs RPM (Approx)</b></caption></table>
<br />
<hr />
<br />steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-14739072541887483592014-06-09T09:24:00.001+10:002014-06-09T09:24:19.822+10:00RAID++, HDD's and OwnershipThe underpinning of Data Storage, especially for large Enterprises, fundamentally shifted when disk drive capacity increase dropped below 7%/year (40% in 5 years, the design-life of HDD's).<br />
<br />
Disk Drives are now <i>consumables</i>, not assets, albeit with a 5-year life. The number of Enterprises with 2,000-5,000 drives in their server room and 10-50,000 drives in their desktop fleet is increasing.<br />
<br />
I posit the following following from this:<br />
<a name='more'></a><br />
<br />
<ul>
<li>Data is the <i>lifeblood</i> of all Enterprises, they are guaranteed to fail if they lose their digital records.</li>
</ul>
<ul>
<li>Data Security, as a right (from state surveillance) and the technical ability to keep data private on- and off-premise, is a requirement of conducting business digitally.</li>
</ul>
<ul>
<li>Drives are constructed of increasingly exotic materials, all in short supply.</li>
<ul>
<li>There is now a requirement for Secure disassembly and material reclamation of drives.</li>
</ul>
</ul>
<div>
In the new world of high-count RAID devices, I suggest the following are necessary within the Drive Industry business model:</div>
<div>
<ul>
<li>Enterprises pay fixed yearly costs per 1,000 drives, covering:</li>
<ul>
<li>replacement of failed drives</li>
<li>yearly supply of replacement drives vs "big-bang" replacement of all drives in units</li>
<li>secure & audited collection and break-down of replaced drives.</li>
</ul>
</ul>
<ul>
<li>Uncontested ownership of Drives and Data used by Enterprises & Individuals.</li>
<ul>
<li>If either drive supplier or customer ceases to trade, the rights to the Data and ownership of drives is incontestable. The supply contracts cannot be <i>leases</i>.</li>
<li>This condition is overwhelmingly important for off-premise drives, particularly backups and archives written over-the-wire and maintained by the Drive Supplier.</li>
</ul>
</ul>
<ul>
<li>Off-site drives, for Continuous Data Protection/Business Continuity, Snapshots, Backups and Archives must be both Secure and specifically identified as <i>owned</i> by a nominated customer.</li>
<ul>
<li>Sharing of drives mustn't be allowed, though multiple customers may share the same RAID array.</li>
<li>This is especially important for seeding Disaster Recovery sites.</li>
<ul>
<li>If necessary, the Preservation Supplier can, overnight, supply new drives containing the Enterprise Data, installed in RAID arrays and ship to the customers recovery site. Easiest if co-located.</li>
</ul>
</ul>
</ul>
</div>
steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-47573747447530770432014-05-26T16:02:00.001+10:002014-05-26T16:15:33.566+10:00RAID and the collapse of IBM's mainframe storage business.What were the technical and non-technical aspects of the collapse in IBM's DASD (mainframe disk) business from 1990 to 1995, from ~$9B and 80% market share to $1.5B and 35%?<br />
<a name='more'></a><br />
Previously: <a href="http://stevej-on-it.blogspot.com/2014/05/raid-timeline-of-ibm-disk.html">a selected timeline</a> of events, some <a href="http://stevej-on-it.blogspot.com.au/2014/05/raid-lessons-from-1990s-market-collapse.html">Lessons on the causes of the collapse</a> and some thoughts on <a href="http://stevej-on-it.blogspot.com.au/2014/05/raid-what-invention-or-change-is-needed.html">RAID++, the next evolution in Data Storage</a>.<br />
<br />
Broadly, IBM DASD's business didn't fall alone, between 1991 and 1993, IBM lost $16 billion, due largely, but not entirely, to problems with their Mainframe business. They were to exit the PC market 10 years later.<br />
<br />
IBM in the early 1960's had "Bet the Company" on a radical new line of computers, the S/360, which went on to dominate the market and become known as "mainframes". At this time, the company was well over 75 years old at the time and already the subject of anti-trust consent decrees, having it's business regulated by the US Government. This wasn't a play by a start-up or an ambitious up and coming technology business: <i>this was an astute and bold gamble by one of the giants in the field</i>.<br />
<br />
It worked and they, and the shareholders, reaped 3 decades of explosive profits and growths. IBM always paid managers and salesmen (mostly Men), handsome bonuses - they got rich as well.<br />
<br />
There were two forces at work that IBM first managed to ride, then managed hold back for around 15 years, postponing the inevitable "Disruption" but at the price of creating a competitive "overhang", that, once triggered, leads to a rapid and profound collapse in the market:<br />
<ul>
<li>high Gross Margins, 60+%, were the norm for IBM to maintain their very expensive and successful sales and marketing techniques, and</li>
</ul>
<ul>
<li><a href="http://research.microsoft.com/en-us/um/people/gbell/Bell's_Law_MSR-2007-TR-146a.pdf">Bells' Law</a>, accounting for the formation, evolution, and death of computer classes based on its technology type. A new computer class forms roughly each decade, establishing a new industry.</li>
</ul>
<div>
<a href="http://research.microsoft.com/en-us/um/people/gbell/Bell's_Law_MSR-2007-TR-146a.pdf">Bell's Law</a>, initially developed in 1972 to forecast the evolution of DEC's PDP & VAX computers:</div>
<blockquote class="tr_bq">
A <i>computer class</i> is a set of computers in a particular price range defined by: a programming environment ... a network; and user interface for communication with other information processing systems including people and other information processing systems. A successful class usually establishes a horizontally structured industry composed of hardware components through operating systems, languages, application programs and unique content ... that serve markets through various distribution channels.</blockquote>
<div>
While IBM's R&D had been increasing capacity & performance while reducing the cost of its DASD and from 1956 onwards (see <a href="http://stevej-on-it.blogspot.com/2014/05/raid-timeline-of-ibm-disk.html">timeline</a> for examples), it's market wasn't the only one developing disk technologies.</div>
<div>
<br /></div>
<div>
The minicomputer and PC markets had been roiling and boiling away since the 1970, growing in volume, revenue and capability every year. No one vendor, like IBM in mainframes, had been able to dominate the market, allowing them to extract massive premiums for their products.</div>
<div>
<br /></div>
<div>
DEC, first with the 16-bit PDP-11 and then the 32-bit VAX lines, <i>was</i> the benchmark and did exceedingly well, but was unable to capture the market and prevent new entrants.</div>
<div>
<br /></div>
<div>
Part of this was due to the arrival in 1969 of UNIX, an Operating System (O/S) from a company barred from selling computers, AT&T. UNIX became transformative because it solved a number of hard Software Engineering problems, especially "scaling up" an O/S. Microsoft failed at this in 2003, just as Multics did in the mid-60's and IBM almost did with OS/360. The IEEE POSIX standard created the formal base needed for the industry to write cheaply portable software.</div>
<div>
<br /></div>
<div>
UNIX was small, well documented and possible to learn independently, <i>portable</i> and performed very well. Linus Torvalds went on to reimplement UNIX from scratch, creating Linux, the O/S adopted by Google for Android. The BSD UNIX distribution fuelled the evolution and development of the system in the days of VAX's and super-minicomputers, ending by becoming the base for OS/X and iOS, the most successful Apple systems ever.</div>
<div>
<br /></div>
<div>
This means that now the most popular Operating Systems on the planet, by units and revenue, are POSIX compliant, i.e. Unix-derived.</div>
<div>
<br /></div>
<div>
First UNIX powered the minicomputer revolution, then Microsoft with MS-DOS then MS-Windows, powered the Intel PC revolution. Without an Operating System and Application Software, hardware isn't useful or interesting to customers. Software sells Hardware and portable Software allows customers to substitute products for others. Software drives competition by vendors and utility for customers.</div>
<div>
<br /></div>
<div>
In 2007 (iPhone) and 2010 (iPad), Apple leveraged their portable POSIX environment to create two whole new Computer Classes, smartphone and tablets, that have seriously impacted the existing PC market and caused financial challenges for Microsoft.</div>
<div>
<br /></div>
<div>
IBM is a remarkable company, not because it failed to see a wave of technological change about to break-over it and destroy one of its core businesses, but because it survived the "market restructure" of mainframes (and PC's) when no other computing business has survived a similar Market Disruption. There is an incredibly long-line of once-great computer companies that no longer exist: DEC and Control Data are two on my list. Burroughs and Sperry-Univac, in the guise of Unisys, are still sold, but the company is around 10% of the size it was in 1990.</div>
<div>
<br /></div>
<div>
By the time of the 1991 crash, IBM could trace a lineage in Data Processing and Storage back 100 years, through a large number of changes in technology: punched cards, mechanical calculators & tabulators, vacuum tubes, transistors, Integrated Circuits, Large Scale Integration and System-on-Chip (SoC) CPU's like Intel's 486.</div>
<div>
<br /></div>
<div>
The collapse of IBM's DASD business can't be viewed in isolation. It was as much financial as technological and market-restructure based and similar collapses affected many of IBM's business lines. The change in the mainframe market cause the second Dept. of Justice anti-trust suit of 1968 to be abandoned in 1982. It took another decade for the full effects of both in-market competition (plug-compatible vendors like Fujitsu/Amdahl) and Bells Law out-of-market substitution (PC's and superminis) to wash through.<br />
<br />
Gordon Bell comments like <a href="http://www.techviser.com/about/bios/johnmasheybio.html">John Mashey</a>, creator of the MIPS chip, that CMOS, the low-power technology used in microprocessors, started in 1971 on its path to replace the high-power ECL technology used in super-computers like Control Data and Cray's, and mid-power Bipolar used in mainframes and high-end superminis. The Cray-3 design was killed off in 1989 because of the impact of CMOS and the collapse in demand for super-computers.</div>
<div>
<br /></div>
<div>
The 1987/8 Berkeley RAID paper was a <i>trigger</i> event, but for anyone who'd worked with both mainframes and minicomputers, the pricing difference was well-known and stark.</div>
<div>
<br /></div>
<div>
EMC because the new dominant player through better business practices and marketing strategies. In 1990, the obvious "Killer App" (or device) for RAID was going to be the StorTek "Iceberg", later taken up by IBM as their RAMAC offering.</div>
steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-14443980313825144722014-05-25T16:51:00.001+10:002014-05-26T16:27:50.475+10:00RAID: Timeline of IBM Disk StorageA highly selective history of Disk Storage <a href="http://stevej-on-it.blogspot.com/2014/05/raid-and-collapse-of-ibms-mainframe.html">referenced later</a>, focussing on IBM's contributions with some of their competition mentioned. This provides context for previous articles, on <a href="http://stevej-on-it.blogspot.com/2014/05/raid-lessons-from-1990s-market-collapse.html">Lessons from RAID</a> and <a href="http://stevej-on-it.blogspot.com/2014/05/raid-what-invention-or-change-is-needed.html">RAID++</a>. A good general reference is the <a href="http://www.computerhistory.org/timeline/?category=cmpny">Computer History Museum</a>. <a href="http://en.wikipedia.org/wiki/Minicomputer">Minicomputers</a> and the laster <a href="http://en.wikipedia.org/wiki/Superminicomputer">Super-minicomputers</a> aren't dealt with, although they created the conditions for the collapse of IBM's mainframe business.<br />
<br />
<br />
<a href="http://research.microsoft.com/en-us/um/people/gbell/">Gordon Bell</a>, while at DEC in 1972, posited "<a href="http://en.wikipedia.org/wiki/Bell's_law_of_computer_classes">Computer Classes</a>" [<a href="http://research.microsoft.com/en-us/um/people/gbell/Bell's_Law_MSR-2007-TR-146a.pdf">PDF, Paper on Bells Law</a>] and how exponential technology growth affects products, markets and eventually companies. Market Disruption from "lower" classes of computers is a feature of computing history. <a href="http://research.microsoft.com/en-us/um/people/gbell/Talkcary/Claws981.ppt">A recent powerpoint by him</a> on the topic.<br />
<br />
In the 1970's, IBM dominated the computer (mainframe) industry generating more revenue that all its competitors combined. They were named "<a href="http://en.wikipedia.org/wiki/BUNCH">The BUNCH</a>": Burroughs, UNIVAC, NCR, Control Data Corporation, and Honeywell.<br />
<br />
In the 1960's it was "IBM and the Seven Dwarfs", until RCA and General Electric sold to Sperry and Honeywell respectively.<br />
<br />
Some items are included on Software Engineering, an on-going challenge in the Industry.<br />
<a name='more'></a><br />
<table><tbody>
<tr><td>1874:</td><td>Thomas J Watson born</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1884:</td><td>Hollerith patent on 80-col punched card: Tabulating Machine Company.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1890:</td><td>US Census, using Hollerith Cards</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1896:</td><td>Thomas J Watson joins NCR, rises to CEO.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1911: </td><td>Computing Tabulating Recording C/o [CTR] formed, merger of 4 c/o, including TMC.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1914: </td><td>Thomas J Watson, leaves NCR, CEO of CTR.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1924: </td><td>CTRC renamed to IBM, International Business Machines</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1942:</td><td>Manhattan Project, "human computers" using calculators, then IBM tabulators. <a href="http://www.mphpa.org/classic/HISTORY/H-06c18.htm">http://www.mphpa.org/classic/HISTORY/H-06c18.htm </a></td></tr>
<tr> </tr>
<tr><td></td><td>John Von Newman instrumental.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1943:</td><td>"There is only a global market for 5 computers". mis-attributed to Thomas J Watson</td></tr>
<tr> </tr>
<tr><td></td><td>ENIAC construction contract signed. Decimal computer.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1944:</td><td>EDVAC construction proposed, binary computer. Von Neumann as consultant, University of Pennsylvania.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1945:</td><td>von Neumann summarized and discussed logical design developments in "First Draft of a Report on the EDVAC".</td></tr>
<tr> </tr>
<tr><td></td><td>Describes "von Neumann architecture".</td></tr>
<tr> </tr>
<tr><td>1946:</td><td>ENIAC commissioned 1946. Moved to Aberdeen, MA 1947. Decomissed 1955.</td></tr>
<tr> </tr>
<tr><td></td><td>Contract signed to build EDVAC for $100,000. Final cost $500,000.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1947:</td><td>Frederick Viehe, amateur inventor, files a core memory patent. IBM later acquired patent rights.</td></tr>
<tr> </tr>
<tr><td></td><td>Transistor invented at Bell Labs (AT&T)</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1949:</td><td>EDVAC delivered to US Army Ballistics Research Laboratory. Delays due to a patent dispute, due to Von Neumanns' 'Report'.</td></tr>
<tr> </tr>
<tr><td></td><td>Harvard physicist An Wang files Core Memory patent.</td></tr>
<tr> </tr>
<tr><td></td><td>US DoJ starts anti-trust case/investigation against IBM. decree in 1956</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td></td><td><hr />
</td></tr>
<tr><td></td><td>End of First Era</td></tr>
<tr><td></td><td><hr />
</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1950:</td><td>First commercial computers built, using Vacuum Tubes (valves) for CPUs. RAM not yet invented.</td></tr>
<tr> </tr>
<tr><td></td><td>LEO - UK, BINAC - USA, Zuse Z4 - Germany.</td></tr>
<tr> </tr>
<tr><td></td><td>RCA’s Jan Rajchman files Core Memory patent.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1951:</td><td>MIT’s Jay Forrester files Core Memory patent.</td></tr>
<tr> </tr>
<tr><td></td><td>An Wang forms <a href="http://en.wikipedia.org/wiki/Wang_Laboratories">Wang Laboratories.</a></td></tr>
<tr> </tr>
<tr><td></td><td>EDVAC commences operation on limited basis. Upgraded in 1953, 54 & 58. Decommissioned in 1961.</td></tr>
<tr> </tr>
<tr><td></td><td>Univac/ERA 1101, vacuum tubes, announced.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1952:</td><td>IBM 701, 36-bit. vacuum tubes, "williams tubes" memory. withdrawn 1971.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM Vacuum Column tape drive</td></tr>
<tr> </tr>
<tr><td></td><td>IRS (US Tax) and Census Data on IBM 701.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1953:</td><td>MIT Whirlwind, 1st computer to use Magnetic Core Memory for RAM.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM 650, Magnetic Drum Memory. 1,800 sold or leased at $200,000-$400,000. 1,000-4,000 words (8.5kB) on Drum + 60 words Core memory.</td></tr>
<tr> </tr>
<tr><td></td><td>Univac/ERA 1103, vacuum tubes. Seymour Cray, b. 1992, involved.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1954:</td><td>IBM 704, vacuum tubes, core memory. 36-bit words.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1955:</td><td>Patent issued, An Wang sells rights for core memory patents to IBM for $500,000.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1956:</td><td>Thomas J Watson Sr, dies. Replaced as Chairman of IBM by TJ Watson Jr.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM 305 computer and RAMAC 350 disk: First Disk Drive. 50 exchangeable platters, 5MB: US$50,000.</td></tr>
<tr> </tr>
<tr><td></td><td>Burroughs B205, a rebranded Electrodata.</td></tr>
<tr> </tr>
<tr><td></td><td>AT&T/Western Electric consent decree: only work in Telecommunications</td></tr>
<tr> </tr>
<tr><td></td><td>IBM consent decree to limit monopoly in Data Processing (punched card). raised 1994 or 1997</td></tr>
<tr> </tr>
<tr><td></td><td>2nd generation transistors developed.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1957:</td><td>John Von Neumann dies.</td></tr>
<tr> </tr>
<tr><td></td><td>FORTRAN developed.</td></tr>
<tr> </tr>
<tr><td></td><td>Digital Equipment Corporation (DEC) founded. In the 1980's, companies in and near Boston on Route 128 formed a hi-tech area like Silicon Valley dubbed the <a href="http://en.wikipedia.org/wiki/Massachusetts_Miracle">Massachusetts Miracle</a>.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1958:</td><td>Sperry Rand's vacuum tube UNIVAC File Computer was one of the first to be used for airline reservations, later Univac 490.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1959:</td><td>Fred Brooks takes over S/360 hardware project.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM 1401, transistor CPU, core memory.</td></tr>
<tr> </tr>
<tr><td></td><td>COBOL developed.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1960:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1961:</td><td>Burroughs B5000 released.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM 7030 Stretch. fastest computer till 1964. At 1.2MIPS, 30 times fastern than IBM 704</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1962:</td><td>SABRE airline reservation system created, on IBM gear.</td></tr>
<tr> </tr>
<tr><td></td><td><a href="http://www-03.ibm.com/ibm/history/exhibits/storage/storage_1311.html">IBM 1311 Model 1 disk released</a>. First removable disk pack, 6 platters, 2MB, 1500RPM. Updated in 1963 & 1968. Most models withdrawn 1971, last 1975.</td></tr>
<tr> </tr>
<tr><td></td><td>Univac 1107, 36-bit words. solid state + thin-film magnetic memory.</td></tr>
<tr> </tr>
<tr><td></td><td>Univac Fastrand Drum bulk storage: ??MB, as 36-bit words. [vs Fastrand II]</td></tr>
<tr> </tr>
<tr><td></td><td>Intergrated Circuits, IC's, as "small scale integration" first sold, displacing discreet transistors.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1963:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1964:</td><td>IBM pays MIT $13 million for rights to Forrester’s patent following years of legal wrangling .</td></tr>
<tr> </tr>
<tr><td></td><td><a href="http://www.computerhistory.org/revolution/memory-storage/8/253">http://www.computerhistory.org/revolution/memory-storage/8/253</a></td></tr>
<tr> </tr>
<tr><td></td><td>S/360 released. 2,000 S/360 on order. OS/360 finished: 5,000 man-years. MVS laster derived from OS/360.</td></tr>
<tr> </tr>
<tr><td></td><td>Control Data Corporation (CDC) 6600. 10Mhz (100ns). x3 faster than 'Stretch'. Fastest computer to 1969, by Seymour Cray.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1965:</td><td><a href="http://www-03.ibm.com/ibm/history/exhibits/storage/storage_2314.html">IBM 2314 disk released</a>, replacing 2311. Withdrawn 1978. Updated model 1970. Removable disk pack, 11 platters, 29.3MB.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1966:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1967:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1968:</td><td>US DoJ starts new anti-trust suit against IBM. dismissed in 1982.</td></tr>
<tr> </tr>
<tr><td></td><td>1st NATO Conference on Software Engineering: 100,000 computers est. in Europe.</td></tr>
<tr> </tr>
<tr><td></td><td>30%-50% Computing Projects 'challenged' or abandoned.</td></tr>
<tr> </tr>
<tr><td></td><td>Univac Fastrand II Drum bulk storage: 100MB, as 36-bit words, or 132M 6-bit characters. 4,500lb, 6' long, 880 RPM, 92msec access, 100KB/sec, Controller: $41,680, Storage unit: $134,400 Max 8 storage units per controller</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1969:</td><td>Thompson and Ritchie, Unix and 'C' on PDP/11. Thompson U. Berkeley graduate, Masters Elec Eng.</td></tr>
<tr> </tr>
<tr><td></td><td>kernel: 20,000 Lines. "Less is More" philosophy in reaction to bloated Multics.</td></tr>
<tr> </tr>
<tr><td></td><td>AT&T barred from computing business. Unix source code licenses sold. mainly 'Educational' to Unis.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM "unbundles" Software and Hardware in reaction to anti-trust suit</td></tr>
<tr> </tr>
<tr><td></td><td>2nd NATO Conference on Software Engineering in Rome.</td></tr>
<tr> </tr>
<tr><td></td><td>DARPA funds "ARPANET" and development of IP Protocols</td></tr>
<tr> </tr>
<tr><td></td><td>Control Data Corporation (CDC) 7600. 36.4Mhz (27.5nsec). Fastest computer to 1976 by Seymour Cray.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1970:</td><td>IBM 3/370 introduced. fully semiconductor.</td></tr>
<tr> </tr>
<tr><td></td><td>SQL developed, IBM's Codd & Date: relational databases.</td></tr>
<tr> </tr>
<tr><td></td><td>Gene Amdah, IBM Fellow (1965), with Fujitsu creates Amdahl Corp. as 'plug-compatible' manufacturer.</td></tr>
<tr> </tr>
<tr><td></td><td>DEC announces PDP-11. 16-bit computer, over 600,000 units sold.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1971:</td><td>8" floppy disk introduced by IBM</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1972:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1973:</td><td><a href="http://www-03.ibm.com/ibm/history/exhibits/storage/storage_3340.html">IBM 3340. "Winchester" disk released.</a> Withdrawn from sale 1983 & 1984. 35MB & 70MB removable disk packs.</td></tr>
<tr> </tr>
<tr><td></td><td>Experimental Ethernet at Xerox</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1974:</td><td>Communications of ACM devoted to Unix.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1975:</td><td>Amdahl ships first S/370 plug-compatible, 470/V6.</td></tr>
<tr> </tr>
<tr><td></td><td>DEC ships first LSI-11, PDP-11 on Large Scale Integration (LSI) chips.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1976:</td><td>Lions Commentary on Unix 6th Ed. Most photocopied book in history, instrumental in Unix.</td></tr>
<tr> </tr>
<tr><td></td><td>Cray-1 by Seymour Cray (left CDC). 80MHz. vector pipelined. 80MFLOPS.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1977:</td><td>UCB aka BSD 1 by CSRB of Berkeley.</td></tr>
<tr> </tr>
<tr><td></td><td>PWB (Programmers Work Bench) from Bell Labs. SCCS version control, shell, troff -mm macros (reused in HTML), make, find/cpio/expr/xargs, yacc/lex</td></tr>
<tr> </tr>
<tr><td></td><td>Rich Miller, Wollongong, 1st port of Unix to Interdata.</td></tr>
<tr> </tr>
<tr><td></td><td>AGSM at Uni of NSW, first commercial Unix license sold.</td></tr>
<tr> </tr>
<tr><td></td><td>DEC VAX-11/780, 32-bit mid-range system introduced.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1978:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1979:</td><td><a href="http://www-03.ibm.com/ibm/history/exhibits/storage/storage_3370.html">IBM 3370 disk released</a>, withdrawn from sale 1986. Sealed Head Disk Assembly, HDA's. removable. 57MB/HDA $35k with controller and $23.4k without. "Fixed Block Architecture" (512-byte sectors). Series 2, 73MB per HDA.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1980:</td><td><a href="http://www-03.ibm.com/ibm/history/exhibits/storage/storage_3380.html">IBM 3380 Model 1 disk released</a>, withdrawn from sale 1986. Non-removable HDA's. Updates released in 1985 and 1987. String of 4x3380 model 1 provided 10GB, at $97.5k-$142k per unit with controllers (1st in string), $81k-$111.5k without. From 1 to 4 drives per unit.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM RISC (Power) introduced</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1981:</td><td>IBM PC introduced.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1982:</td><td>IBM share of mainframe sales 62%, down from 70% in 1974 (or 1969?)</td></tr>
<tr> </tr>
<tr><td></td><td>DoJ anti-trust suit of 1968 dismissed</td></tr>
<tr> </tr>
<tr><td></td><td>Judge Greene hands down AT&T consent decree, breaking up Bell System and removing 1956 constraints.</td></tr>
<tr> </tr>
<tr><td></td><td>Cray-XP, multi-processor, released. not by Seymour Cray.</td></tr>
<tr> </tr>
<tr><td></td><td>Ethernet II, 10Mbps over thick coax</td></tr>
<tr> </tr>
<tr><td></td><td>SUN Microsystems, 32-bit Unix workstations, founded.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1983:</td><td>ACM Turing Award for Unix to Thompson & Ritchie.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM PC-XT, 5MB disk, $4,995.</td></tr>
<tr> </tr>
<tr><td></td><td>Ethernet 10BASE5, 10Mbps over thick coax, as IEEE 802.3 standard.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1984:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1985:</td><td>Cray-2, 125 MHz (8nsec), 1900 MFLOPS. Not commercially successful - hard to attain max perf. Fastest computer till 1990 (ETA-10). ECL logic.</td></tr>
<tr> </tr>
<tr><td></td><td>Ethernet 10BASE2, 10Mbps over thin coax.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1986:</td><td>Sperry-Univac and Burroughs merge to form Unisys. #2 and #3 in Mainframes. revenue of $10.5 billion in 1st year, but went into decline, dropping out of S&P 500 in 2008.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1987:</td><td>RAID Paper, Patterson, Katz & Gibson, U. Berkeley.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM PS/2 closed technologies: OS/2, MCA (microchannel architecture). OS/2 Warp v. late, failed.</td></tr>
<tr> </tr>
<tr><td></td><td>PS/2 keyboard and mouse connector becomes standard until replaced by USB</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1988:</td><td>IBM 3380/3390: $200,000 for 11GB-22GB. Avg purchase: $750,000 for 3 devices</td></tr>
<tr> </tr>
<tr><td></td><td>IBM has 80% of $11.4 billion Mainframe Storage market.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1989:</td><td><a href="http://www-03.ibm.com/ibm/history/exhibits/storage/storage_3390.html">IBM 3390 Model 1 released.</a> 3.78GB to 11.35GB per unit at $90k-$275k/unit.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1990:</td><td>EMC Symmetrix 4200, 24GB, RAID-0, $200-$300,000.</td></tr>
<tr> </tr>
<tr><td></td><td>Intel RAMBUS memory. failed, DDR won.</td></tr>
<tr> </tr>
<tr><td></td><td>Ethernet 10Base-T standardised, 10Mbps over "cat 3" twisted pair.</td></tr>
<tr> </tr>
<tr><td></td><td>Last DEC PDP-11 models /93 and /94 introduced. discontinued 1996.</td></tr>
<tr><td></td><td>An Wang dies, his son, Fred Wang, succeeds him as President Wang Laboratories.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1991:</td><td>Thompson et al: Plan 9, Enterprise, portable, extensible, secure O/S</td></tr>
<tr> </tr>
<tr><td></td><td>Linus Torvalis, Linux 0.01. Terminal emulator.</td></tr>
<tr> </tr>
<tr><td></td><td>Microsoft release Windows 3.1.1 with LAN support</td></tr>
<tr> </tr>
<tr><td></td><td>Intel releases 486, first single-chip 32-bit full computer (ALU, FPU, Virtual Memory, cache)</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1992:</td><td>IBM posts $8B loss, largest in US Corporate History</td></tr>
<tr> </tr>
<tr><td></td><td>IBM releases 1st 'Thinkpad' laptop</td></tr>
<tr> </tr>
<tr><td></td><td>DEC releases "Alpha" 64-bit RISC processor.</td></tr>
<tr> </tr>
<tr><td></td><td>Wang Laboratories files for Chapter 11 (bankruptcy) protection</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1993:</td><td>From 1991 to 1993, IBM posts $16 billion is Corporate Losses. Louis V. Gerstner appt. CEO.</td></tr>
<tr> </tr>
<tr><td></td><td>IBM 3390 Model 3 released.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1994:</td><td><a href="http://en.wikipedia.org/wiki/Linux_kernel">14 March 1994, Linux 1.0.0 released</a>, 176,250 lines of code.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1995:</td><td>BSD 4.4, final release from CSRG @ UCB.</td></tr>
<tr> </tr>
<tr><td></td><td>Cray Research files for bankruptcy. </td></tr>
<tr> </tr>
<tr><td></td><td>"Fast" Ethernet 100Base-T standardised, 100Mbps over "cat 5" twisted pair.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1996:</td><td>PDP-11 series discontinued.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1997:</td><td>Amdahl becomes fully owned subsiduary of Fujitsu.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1998:</td><td>DEC acquired by Compaq.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>1999:</td><td>Ethernet 1000Base-T standardised, 1Gbps over "cat 5a" & "cat 6" twisted pair.</td></tr>
<tr> </tr>
<tr><td></td><td>Wang Laboratories acquired by Getronics, of Netherlands.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2000:</td><td>Compaq discontinue DEC's VAX-line. 2005 manufacture discontinued.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2001:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2002:</td><td>Compaq merges with Hewlett-Packard (HP)</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2003:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2004:</td><td>Microsoft "Longhorn Reset". 2.5 years work by 10,000 ppl discarded. 50M Lines of Code?</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2005:</td><td>IBM sells Personal Computing Division to Lenovo.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2006:</td><td>Ethernet 10GBase-T standardised, 10Gbps over "cat 7" twisted pair. Not widely taken up</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2007:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2008:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2009:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2010:</td><td>40GBase-x & 100Gbase-x standardised. 40Gbps and 100Gbps over Optical Fibre.</td></tr>
<tr> </tr>
<tr><td></td><td>SUN Microsystems, acquired by Oracle Corp.</td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2011:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2012:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td>2013:</td><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
<tr><td></td></tr>
<tr> </tr>
</tbody></table>
<br />
<a href="http://stevej-on-it.blogspot.com/2014/05/raid-and-collapse-of-ibms-mainframe.html">Later summary of Market and Technological forces </a>and the causes of the collapse.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-15673079042593650372014-05-23T18:48:00.001+10:002014-05-26T13:33:37.185+10:00RAID++: Lessons from the 1990's market collapse for Large Expensive Disk DrivesFollowing<a href="http://stevej-on-it.blogspot.com/2014/05/raid-what-invention-or-change-is-needed.html"> a previous post</a>, this is an examination of the causes of the collapse in IBM's Mainframe DASD (Direct Access Storage Device) business, from an 80% share of an $11.B market in 1990, to ~25% a decade later. (<a href="http://faculty.insead.edu/adner/projects/Data%20storage%20project_vfinal.PDF">Source</a> PDF). Notably, IBM's DASD business wasn't the only line to suddenly collapse: the Mainframe business as a whole collapsed, creating a loss across the whole business, the first since 1914.<br />
<a name='more'></a><br />
<br />
<table border="1px solid black" cellpadding="5px" style="border-collapse: collapse;"><caption align="bottom"><br />
Mainframe market share 1991 – 1998: % total Terabytes shipped</caption> <colgroup> <col span="9"></col> <col style="background-color: #e7e7e7;"></col> </colgroup> <thead>
<tr><th></th><th>1991</th><th>1992</th><th>1993</th><th>1994</th><th>1995</th><th>1996</th><th>1997</th><th>1998</th><th>Avg. ∆ / year</th> </tr>
</thead><tbody>
<tr><td><b>EMC</b></td><td>2</td><td>5</td><td>17</td><td>32</td><td>40</td><td>42</td><td>48</td><td>51</td><td align="center">+59%</td></tr>
<tr><td><b>IBM</b></td><td>66</td><td>61</td><td>48</td><td>36</td><td>36</td><td>29</td><td>30</td><td>27</td><td align="center">-12%</td></tr>
<tr><td><b>Hitachi</b></td><td>22</td><td>23</td><td>25</td><td>19</td><td>11</td><td>18</td><td>18</td><td>18</td><td align="center">-3%</td></tr>
<tr><td><b>StorTek</b></td><td>2</td><td>1</td><td>0</td><td>5</td><td>10</td><td>9</td><td>0</td><td>0</td><td align="center">N/A</td></tr>
<tr><td><b>Amdahl</b></td><td>8</td><td>10</td><td>10</td><td>8</td><td>3</td><td>2</td><td>4</td><td>4</td><td align="center">-9%</td></tr>
<tr bgcolor="lightgrey"><td><b>Total TB</b></td><td>-</td><td>-</td><td>703</td><td>906</td><td>1,581</td><td>2,546</td><td>3,367</td><td>5,019</td><td align="center">+48%</td></tr>
<tr><td><b>Total $MM</b></td><td>-</td><td>-</td><td>4,467</td><td>3,937</td><td>4,176</td><td>3,684</td><td>3,121</td><td>2,964</td><td align="center">-8%</td></tr>
<tr bgcolor="lightgrey"><td><b>Avg $k/GB</b></td><td>-</td><td>-</td><td>6.35</td><td>4.34</td><td>2.64</td><td>1.45</td><td>0.93</td><td>0.59</td><td align="center">-38%</td></tr>
</tbody></table>
<br />
EMC and Storage Technology (StorTek or STK), <a href="http://stevej-lab-notes.blogspot.com.au/2011/11/building-raid-disk-array-circa-1988.html">took different paths to releasing Mainframe RAID products while NCR focussed on Unix/Mini computer products with a SCSI device priced $20,000-$100,00.</a><br />
<br />
EMC and StorTek both already built Mainframe 'Channel' attached devices (e.g. ESCON) and a sales presence at customer site. Stortek's 9200 Iceberg may have been released in 1994 (or 1991 above), while EMC's 4200, a much smaller and simpler model, was for sale in late 1990. In 1996, Stortek and IBM signed an agreement to resell the Iceberg exclusively as IBM's RAMAC.<br />
<br />
EMC first model wasn't a high-reliability RAID, but was small, fast, more expensive per GB than IBM's 3390 and targeted at a very specific niche of users. It released new models ever year, increasing features and capacity. It leveraged its first mover advantage to good effect.<br />
<br />
The Mainframe market was demanding with Performance and Reliability/Availability were prime metrics, with per-unit capacity and Price per GB were less important.<br />
<br />
RAID only became possible in the Mainframe market because of three developments elsewhere:<br />
<br />
<ul>
<li>The 1987/8 Berkeley paper on RAID providing, without patent licensing restrictions, the theoretical base for RAID-5 and 6, allowing arbitrarily large capacity, reliability or performance (random IO/sec) devices to be built, albeit at a premium. Operationally, these were compelling reasons.</li>
</ul>
<ul>
<li>The rise of micro- and mini-computer (or low- and mid-range) markets, with a strong market of independent Storage vendors charging normal commercial margins of 20%-30%, not the "premium-priced" Mainframe products with 50%-70% gross margins. The competition between vendors grew both the size of the market, kept margins down and increased the rate drive capacity increased, as per-unit input costs were relatively static, pushing vendors to seek higher storage densities.</li>
</ul>
<ul>
<li>The creation of Industry Standards for disk sizes & connectors and disk controller interfaces:</li>
<ul>
<li>SCSI series for mini-computers. Initially 5Mhz and 8-bits wide for 40Mbps transfers using standard connectors, and</li>
<li>IDE (now called ATA) as the common disk interface.</li>
<li>Specialist vendors, such as Adaptec, built disk controllers for many types of computers and different drive interfaces, IDE and SCSI. </li>
<li>The two drives mentioned in the Berkeley 1987/8 paper, the larger 600MB Fujitsu 'Super Eagle' were 5.25", and smaller PC drives, such as the 100MB Conner Peripherals were 3.5".</li>
<ul>
<li>As the PC market expanded, demand for 3.5" drives, by units and total capacity, rose dramatically faster than all other segments.</li>
<li>Increased volumes and intense competition drove the PC-drive vendors to improve products, yield, capacity, performance and prices through automation and improved production facilities.</li>
<li>Around 1995, 3.5" drives supplanted 5.25" drives in RAID arrays.</li>
</ul>
</ul>
</ul>
<div>
Mainframe vendors, specifically EMC and Stortek, saw the potential for the potential for substituting low-cost, low-performance and lower-reliability mass-market drives for the expensive 'DASD' drives sold by IBM.</div>
<div>
<br /></div>
<div>
We know from NCR and HP's efforts in the low and mid-range markets that those clients also valued new features that RAID could bring then, even though it was more expensive per GB because low- and mid-range computers were already directly using the 3.5" and 5.25" assembled in RAID arrays.</div>
<div>
<br /></div>
<div>
Data General's Clarion line of RAID devices for mid-range (Unix) systems was acquired by EMC within the decade, allowing them to access that market segment as well.</div>
<div>
<br /></div>
<div>
There wasn't a widespread, strong demand in any market segment for all the benefits provided by RAID. However, in almost all market segments, there was one or two compelling features of RAID that compensated for the additional cost and complexity of RAID arrays, allowing the large vendors to continue with 50+% gross margins for some time.</div>
<div>
<br /></div>
<div>
The Mainframe segment is not mostly price-determined. While not insensitive to price, they are sophisticated consumers with deep pockets, taking time to assess vendor offerings against each other on multiple criteria and able to gain deep discounts on published prices. While we can see the aggregated sales figures for EMC et al, we don't know the discounts they offered to major clients to win sales.</div>
<div>
<br /></div>
<div>
In a "premium-priced" market, the idea of deep discounting seems odd, but is a sales tactic used by many vendors.</div>
<div>
<br /></div>
<div>
<b>Summary</b>:</div>
<div>
<ul>
<li>The replacement of SLED's by RAID in the Mainframe market was fast, and for IBM bloody: an archetypal Christensen "<a href="http://www.claytonchristensen.com/key-concepts/">Disruptive Innovation</a>".</li>
<ul>
<li>This sudden change was primarily due to commercial, not technological factors.</li>
<li>EMC conclusively demonstrated that RAID's competitive advantage wasn't just <i>price</i>, either per-unit or per GB.</li>
</ul>
<li>The technological conditions creating the base for RAID took nearly 2 decades to build. They started with the competition between mini-computer vendors (e.g. DEC) of the early 1970's forcing innovation across all their product lines.</li>
<li>Two necessary critical conditions were:</li>
<ul>
<li>The license-free Berkeley RAID design of 1987/8, and</li>
<li>the ramp-up in IBM pricing on all its Mainframe products during the 1980's prior to the 1991 collapse in profits,</li>
</ul>
<li>The large and vibrant market in low- and mid-range computers, pushing innovation and preventing excessive margins.</li>
<li>The existence of multiple "plug-compatible" vendors in the Mainframe market, with established sales channels and good reputations, and that could quickly develop new products.</li>
<ul>
<li>The development of high-end RAID products helped create a RAID market for mid-range computers.</li>
</ul>
</ul>
<div>
The IBM management team that created this perfect storm during the 1980's cannot be criticised: they followed the MBA "milk the cash-cow" strategy <i>perfectly</i>.</div>
<div>
Only with hindsight can anyone criticise their actions and decisions: from the data available to them at the time, there was nothing to indicate they should be acting differently. Clayton Christensen and others have written extensively on this effect.</div>
</div>
steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-71827217943288580022014-05-22T13:12:00.003+10:002020-12-11T08:42:49.240+11:00RAID++: What invention or change is needed now to take us the next 25 years?For an industry, digital data storage, that's seen disruptions every 25 years, we're either overdue for one or The Next Revolution has arrived, but nobody's noticed.<br />
<br />
What will the Next Big Thing in Storage look like? What, if anything, will succeed RAID, if it hasn't already? Are their lessons we can learn by examining the last big disruption circa 1990 in Storage: RAID arrays?<br />
<br />
The one certain lesson from 1955 and 1990 is that <i>nobody</i> can guess, not even remotely, what Data Storage will look like in 25 years time. Any prediction of the 2040 market will be wildly inaccurate, but we can talk about current market forces and technologies and where the trends point for the next 5 and 10 years.<br />
<a name='more'></a><br />
In 1956, IBM released the first magnetic <i>disk,</i> not 'drum', drive, the RAMAC 350, providing 5MB via 50x24" swappable disks. 1,000 of these units were sold for $50,000 each. In today's money, that's around $80 million per Gigabyte, compared to the current $0.04 to $0.12 per GB. A 100 million times reduction in price over 5.5 decades, or 5-doublings every decade. The price per bit averaged a halving every two years: a most impressive technological achievement.<br />
<br />
In terms of the $100,000- $500,000 per unit for Enterprise Storage from 1950 to 2000, sub-$100 drives and cents per GB, render current raw storage effectively Free and Infinite.<br />
<br />
What's changed in the last 10 years is the <i>rate</i> that Drive size increased and hence Price per GB dropped, has slowed from 100%/year at its peak to 7%-12% now. The economic policies of yesteryear can no longer work: "Wait 3 years and you can store everything you have now, for less" no longer applies. In 5 years, you might be lucky to pay less per GB. The floods in Thailand showed that supply is not a given and future prices can't be programmed in.<br />
<br />
There has also been an explosion in the number of identifiable market segments (from one for all computers in 1956), each with segment having different product demands and price-points. The embedded device 1.8" drive market is unrelated to mobile computing which is unrelated to Enterprise Storage, the modern name for 1980's "mainframes" and high-end Unix computers.<br />
<br />
The collapse since 2007 in rate of capacity increase is compounded by the increasing speed of Internet connections, with a continued high, exponential, rates of both data downloaded and stored. At exactly the time customers and the Industry need the yearly 100% expansion in capacity, it isn't available.<br />
<br />
The most startling insight for many is that Data Storage devices, domestic, commercial, Enterprise and Cloud <i>are not capital items</i>. Storage isn't a Balance Sheet <i>Asset</i>, it's a <i>consumable</i>, albeit with a 5-year life. Drives vendors understand this, but customers seldom act this way.<br />
<br />
Technology cannot be separated from Competitive forces and Commercial drivers: Computing and Information Technology is primarily a <i>business</i>, not an Engineering or Research Discipline.<br />
<br />
Successful I.T. technologies are driven by <i>money</i>, not the engineering. It's also how they fail.<br />
<br />
What customers <i>will</i> buy and how much they are prepared to pay for it at any one time completely determines the market price and volumes.<br />
<br />
It's a demand-side business: <i>supply</i> doesn't drive purchases, consumer <i>demand</i> does.<br />
Purchases are driven by customer need, price-point and compatibility with existing systems and staff capability. Data Storage is only a minor component of whole Computing/I.T. costs for Enterprises of all sizes. Building "a better mouse-trap" in this market won't yield one sale unless you can convince customers it meets a real need of theirs at an attractive price.<br />
<br />
Storage devices can't be compared solely on purchase price, they can only be compared through Total Cost of Ownership, TCO, over the complete purchase, install, operations, migration, replacement and disposal process, <i>with</i> backups, archives and DR Plan protection/recovery included.<br />
<br />
Given the turnover in devices and increasing demand for Data Storage, every site now has to cope with constant upgrades. The major factor in purchases should now be: <i>How much can I grow this unit and is that enough for the next 3-5 years? How much will I have paid in 5 years time,</i> when I come to replace this unit?<br />
<br />
The correct accounting option for comparing Data Storage is: <i>yearly TCO, per GB or IO/sec or another metric valued by the customer.</i><br />
<br />
While it may cost $100-$500 to buy a single drive, <i>using</i> that drive will cost 5-10 times as much over its lifetime. Adding the resources to provide reliable access to that data may raise the costs by another factor of 10. Being a Digital Packrat and never cleaning up and properly housekeeping Data is becoming prohibitively expensive.<br />
<br />
We know from PCIe-Flash cards (Fusion-IO) and "Pure Flash" Enterprise Devices that the high-end market is prepared to compare on TCO and whole system performance, not price per GB.<br />
Raw "Flash", or NAND EEPROM, has around 10-times the raw price per GB of current Hard Disk Drives, HDD's, yet they are winning substantial sales.<br />
<br />
This suggests two things:<br />
<ul>
<li>The fully-loaded systems cost per GB and cost per IO/sec of HDD storage is higher than Flash. HDD's, when built into large RAID Arrays aren't cheap, they're between $1-$10/GB not the raw cost of $0.10/GB.</li>
</ul>
<ul>
<li>Cost per GB is not a useful prime comparison metric anymore for most users.<br />Enterprise Data Storage isn't capacity constrained now.</li>
</ul>
The RAMAC's performance in 1956 was slow by todays' standards: 600 msec access times, 10kbit/sec transfer and 1,200RPM<br />
versus current 2-4 msec seek times, 1-1.5Gbps transfer rates and 5,400 - 15,000 RPM.<br />
<br />
Disk <i>performance</i> has improved by 10-100 times for access and for transfer rate by 100,000 times, changing the <i>performance balance</i> of drives radically. This shows up in the time it takes to "scan", or read/write, an entire drive. A 4TB drive takes ~10 hours to read/write in 'streaming' mode, but 75 - 150 hours via random read/writes at 64KB. This longer time is the rebuild time for a failed drive in a RAID array, without allowing for any data updates (RAID-5 & 6 carry a 3-6 fold penalty to write new data), any other Application access or any congestion of the internal hardware and links of the RAID array. Real world experiences of RAID arrays taking over 1 week to rebuild a failed drive, and losing half the performance for the duration, are not uncommon. This is a problem because RAID-5 is very fragile: if there is even <i>one</i> read error during this time, then you lose <i>all</i> the data in that Volume. It doesn't event take a second failed drive to lose the lot.<br />
<br />
Disk Drives, or more properly, Digital Data Storage, have many characteristics of interest to customers, besides the basic performance specs: {capacity, sustained transfer rate, latency or seek time and its inverse, random IO per sec}.<br />
<br />
Reviewing the literature, historic technical specifications and industry press, the Industry, both Vendors and Consumers, still seem to be solely focussed on Price and Capacity, either as per-unit or Price per GB, as the prime comparative metric for Data Storage.<br />
<br />
There are many other operational dimensions, besides Performance and Price. Around 1995, all large businesses had become completely dependent on Computers/I.T. for back-office systems, by 2005 and the Internet Revolution, this had transitioned to Front-Office Operations for large businesses.<br />
<br />
Reliable Data Storage isn't optional anymore in a 24/7, fully on-line world. Your GB <i>have</i> to work, <i>have</i> to be complete & correct and <i>have</i> to be available, or your business will suffer badly, even fail.<br />
<br />
With the advent of Mobile Devices, Cloud Computing and affordable, ubiquitous Virtual Machine environments, this trend has pushed even further. For many markets, like portable MP3 players, Flash at $1/GB has become the only economic solution, forcing consumers to look for alternate means, like Cloud Storage, to meet their Reliability and Data Protection/Archive needs. Here are some of the Dimensions now considered when purchasing Digital Data Storage:<br />
<ul>
<li>Unit Price and Systems Price (includes racking, controllers, cables & connectors, switches and ancillary/support equipment like power-supplies, fans etc)</li>
<li>Power and Cooling needs,</li>
<li>Machine Room Footprint or Device Volume.</li>
<ul>
<li>Large, heavy devices needing 200-tonne cranes to lift into place are no longer saleable.</li>
</ul>
<li>Duty Cycle: The Load Factor per hour, day or week the drive is expected to handle.</li>
<li>Drive Reliability: Mean Time Between Failures (MTBF)</li>
<li>Media Durability: Expected Media/Device Lifetime in Hours or Operations.</li>
<li>Data Protection: Mean Time to Data Loss</li>
<li>Data Durability: Mean Time to 5% Media Degradation</li>
<li>Data Reliability/Certainty: Undetected Bit/Block Error Rate (UBER) and </li>
<li>and many more for specific Applications and Environments:</li>
<ul>
<li>Weight and Floor-loading. Over-weight devices can't be used in commercial buildings.</li>
<li>Disaster Recovery, Business Continuity and Contingency Planning constraints and interactions.</li>
<ul>
<li>If a site is destroyed, how quickly can the business be trading again and to be fully operational?</li>
<li>What is the additional cost for Continuous Data Protection and off-site replicas?</li>
</ul>
<li>Media replacement costs, Off-load, Upgrade and Migration options:</li>
<ul>
<li>Can the device be upgraded in place? Is it sealed? Must it be replaced 'with a forklift', not upgraded in-place.</li>
<li>Floppy disks wore out in use and writable Optical Disks faded, creating a yearly consumables budget. Do new technologies, like USB Flash drives, work the same?</li>
<li>If it takes 3 months to load and off-load all data on a device, its unusable for most clients.</li>
<li>If you're an Airline or even a large on-line business with a 24/7 zero-break operation, how do you migrate to a new datacentre or install new Storage?</li>
</ul>
<li>Special Operational Requirements, such as Cryogenic Cooling, refrigeration/coolant or high temperatures.</li>
<li>Multiple suppliers, to avoid "lock-in" and monopolistic price-gouging. [The IBM 'SLED' customer experience.]</li>
<li>Support and Supply: Can the vendor supply product all demand with low & predictable lead times, support the products customers purchase and are enough trained operators and admins available to early or late adopters?</li>
<li>Extended Vendor support and replacement: some applications demand <i>no</i> uncertified hardware changes in a system service life of 25+ years.</li>
<li>Quality as consistency of performance & durability for demanding applications.</li>
<li>Robustness against change in hostile environments, like Space or Deep Sea.</li>
<li>Endurance in extreme environments: very hot, very cold, very wet or dry or high G-force or extreme vibration. [Mobile devices are 'challenging' for normal HDD's.]</li>
<li>Media Longevity for Archival uses (100+ years)</li>
<li>Security and Safety: can data be read or copied from discarded media/devices, or can the device catastrophically fail and injure people? Exploding drives is a bad look.</li>
<li>Usability, Operability, Manageability: Is the hardware cheap <i>in use</i>? Is it compatible with existing systems, applications and other hardware?</li>
<li>Scalability and Inter-device interference: How many devices can I put in a single enclosure, while still being able to do in-place, live upgrades?</li>
</ul>
</ul>
It's beyond the scope of this piece to fully examine the causes of the last Big Storage Disruption in 1990, the rise of (Enterprise) RAID Arrays.<br />
<br />
IBM, for almost 3 decades from 1964 with it's S/360-architecture, had 60% by Revenue of the Computing market. In the 1970's, the "mini" computer arrived, creating a whole new low-cost market and putting pressure on its traditional business. IBM introduced the PC XT in 1984, leading to real pressure on both the mainframe and mini-computer businesses. The 1980's also saw the arrival of high-end "minis" and RISC, with the rise of the DEC VAX and SUN's SPARC, used extensively for databases and business applications.<br />
<br />
The S/360 line, a revolutionary idea at the time, was years late and famously over-budget. It was truly a "bet the company" decision by the IBM Board, just as the 747 was later for Boeing, arriving in 1969. Both companies, against the odds, succeeded, and succeeded beyond the wildest dreams of managers, shareholders and the salesmen on quotas and bonuses. Sometimes big business bets pay off, but that's the exception, not rule.<br />
<br />
By the time the RISC research group at Berkeley published their RAID paper in 1987, IBM had ~80% of an $11.5B/yr Mainframe Storage market. The 3380 line cost $25-$50,000/GB in 1988. It's successor, the 3390 was released in 1989 for around $25,000/GB, twice the cost that EMC ended up charging for its Symmetrix 4200 line, based on 1GB, 5.25" Seagate ST4120 SCSI drives, costing $3-4,000/GB.<br />
<br />
EMC's first 24GB RAID-0 (no protection) systems sold for $200-$300,000 and were more expensive than IBM's 3380/90 drives, though considerably faster and smaller (by a factor of <i>five</i>) and consumed a small fraction of the power, providing significant savings in machine room floorspace and avoiding cooling upgrades being forced by more power-hungry CPUs. EMC fuelled its growth through high Gross Margins (60+%), strong product development (becoming a product leader and consistently first-to-market), diversifying into other markets, forming supplier and reseller alliances and very good strategic acquisitions. Mainly, they succeeded because they listened very closely to consumer concerns and demands and responded quickly to those concerns.<br />
<br />
The Berkeley paper had dubbed existing mainframe drives "SLEDs", Single Large <i>Expensive</i> Drives, and invented the acronym, RAID, as Reliable Array of <i>Inexpensive</i> Disks, later changed by the industry to <i>Independent</i>, as no high-end products were 'inexpensive', nor would high-end customers buy 'cheap', they wanted Fast, Good and Reliable.<br />
<br />
The drivers of the 1990 Storage Disruption were two-fold:<br />
<ul>
<li>a halving in price per GB, with a massive lift in performance and operational factors, and</li>
</ul>
<ul>
<li>Radically increased Reliability and hence better Data Protection. </li>
<ul>
<li>Seagate drives had 40,000hr MTBF and IBM 3380/90 more like 52,000hrs (vs the 800,000-1.2M of current drives). [11-dec-2020. KATZ et al, 1989 "Introduction to redundant arrays of inexpensive disks"]</li>
<li>But with RAID-1 or RAID-5 and hot-spares, EMC could effectively deliver an near infinite MTBF.</li>
</ul>
</ul>
By 1994, EMC had overtaken IBM as the dominant vendor (~35% each), in a market that had shrunk significantly, in revenue terms, and was to continue to shrink to half its original size, leaving IBM with closer to 10% and EMC nearer 50% market shares. IBM did attempt to counter the RAID phenomena, by partnering with Storage Technology (StorTek, STK) rebadging their "Iceberg", then later signing an exclusive redistribution deal with STK, only to be challenged by the US Justice Dept. as anticompetitive and monopolistic.<br />
<br />
From 1991 to 1993, IBM declared losses of $16B, with $8B in one year, the largest corporate loss in US history to the time. This was IBM's <i>first</i> loss since the 1914 merger that created IBM. From the 1964 introduction of the S/360, IBM had made <i>record profits</i> every year. The sudden and unanticipated collapse of IBM's business model took them and the business world by surprise. The sudden collapse of their mainframe Data Storage business didn't help their overall financial results, but when combined with a large-scale market rejection of overly high charges across all lines of IBM's business, it led to a massive co-ordinated business disaster for them. For over a decade, IBM had been unwilling to cannibalise it's own lines of business or to meet the market and reduce its Gross Margins. IBM managers had, they thought, a captive market and could "dial-in" the years profits beforehand. As customers fled in increasing numbers, the prices charged to the smaller remaining group had to increase faster and faster, until finally customers found "substitutes" faster than IBM could raise prices.<br />
<br />
The collapse of the mainframe SLED market cannot be viewed in isolation from the financial implosion of IBM. IBM's recovery wasn't certain, a lesser management team would've failed.<br />
<br />
EMC did everything right, following exactly the path IBM took from 1950 to 1964: listening to the customer, innovating, being first-to-market, selling by "Business Benefit" not "technical numbers" and, most importantly, maintaining high Gross Margins while still fitting in with customer price-points. It then reaped the benefits of an early lead for the last two decades.<br />
<br />
EMC has continued to diversify, buying companies like VMware, the innovator and market leader in the Intel x86 Virtual Machine segment. They are far from a spent force, but are they following IBM to a sudden collapse in their market? That doesn't ever show up in the preceding Annual Reports.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-3840282109395319802014-03-30T18:13:00.001+11:002014-04-12T13:05:05.203+10:00The New Disruption in ComputingSince 2000, we've been progressively bumping into limits, or end-points, in all areas of silicon technologies. <a href="http://stevej-lab-notes.blogspot.com.au/2011/12/revolutions-end-computing-in-2020.html">Some I've mentioned previously.</a><br />
<br />
The thesis of this piece is the Next Technology Disruption is <i><a href="http://stevej-on-it.blogspot.com.au/2014/03/the-new-disruption-in-computing.html#1">No Disruption, No Revolution:</a></i><br />
<blockquote class="tr_bq">
instead of exponential growth of technologies, decreasing unit prices and increasing volume sales, we're now seeing zero or slow growth, steady or increased unit prices (especially if supply-chain is disrupted), and in all but a few market segments, sales are in decline and profits are stressed in many vendors.<i> I believe these are linked.</i><br />
<a name='more'></a></blockquote>
The list of Technology Roadblocks since 2000 is long and deep:<br />
<ul>
<li>around 2002, Moore's Law for single CPU's hit the Heat/Power Wall, forcing Intel & AMD into multi-core designs.</li>
<ul>
<li>Laptop sales, by units, overtook Desktop sales in 2008.</li>
<li>Since the 2007 introduction of the iPhone and 2010 introduction of the iPad (and 2008 Android phone, then tablet), sales of PC's.</li>
<li>In 2013, sales of Tablets, by units, had outstripped all PC sales.</li>
<li>Sales of Intel and AMD</li>
</ul>
</ul>
<ul>
<li>In 2005, DRAM was dropped as the reference Technology step</li>
<ul>
<li>Flash memory kept doubling in capacity and halving price-per-bit at historic rates.</li>
<li>This has seen the death of both Floppy Drives and Optical Disks (CD, DVD's) in consumer products.</li>
</ul>
</ul>
<ul>
<li>In 2010, Hard Disk Disks (HDD's) had missed their evolution targets. 4TB had been forecast while 2TB shipped.</li>
<ul>
<li>Small HDD's and Floppy disks have been killed by USB Flash drives and SSD. At around $1/GB for Solid State Disk (SSD), drives under $50-$75 are only SSD's.</li>
<li>2013 saw sales of HDD's, by unit, decline 5%, the second consecutive year of contraction.</li>
</ul>
</ul>
<ul>
<li>I have no information on the evolution and pricing of GPU, Graphics Processors and Video Cards, nor Monitors. We seem to arrive at 30 inch displays with 2560 resolution 5-7 years ago.</li>
</ul>
<ul>
<li>General use of Optical Drives plateaued at DVD single-sided capacity, 4.7GB.</li>
<ul>
<li>Bluray disks, 30GB (?), are sold in relatively limited volumes and media is comparatively rare and expensive.</li>
<li>Commodity consumer-grade SD-cards, used in laptops, video cameras, still camera and phones & tablets are available everywhere and come in sizes from 4GB to 32GB. At around $1/GB, they've relegated write-once Bluray to the technology off-ramp.</li>
</ul>
</ul>
<ul>
<li>Ethernet via Twisted Pair, erupted in the early 1990's with 10Mbps and by 1999 was up to 1Gbps. It's stagnated since.</li>
<ul>
<li>The 'ethernet' market is estimated to be a $16B/year business.</li>
<li>Today every PC (Desktop and Laptop) with an ethernet port supports 10/100/1000Mbps speeds. </li>
<li>In 2002, a 10Gbps standard was released, but took until 2007 to ship 1M ports. Although a twisted-pair 10Gbps standard exists, it is very rarely seen and hasn't appeared in common consumer devices.</li>
<li>In 2010, fibre only updates to Ethernet were released, 40Gbps and 100Gbps. These are especially important for server rooms and long-distance links.</li>
<ul>
<li>DWDM, Dense Wave Division Multiplexing (or 'Many Colours'), allows 64 or 96 signals to be carried on one fibre. The cost of electronics, and even fibre, is tiny compared to the cost of civil works installing fibre over 10 kilometres.</li>
</ul>
</ul>
</ul>
<br />
The one bright point in the Technology landscape is in the Wide Area Network (WAN). Fibre to the Premises (FTTP), is starting to provide more access to affordable, guaranteed 100Mbps and 1Gbps services to all customers. This will drive new applications in business market: branch offices, SOHO and SME's.<br />
<br />
<b>Economics of Volume Production</b><br />
<br />
Since the IBM PC was released in 1981, the economics of Volume Production has driven the Hardware and Software world. When the cost of R&D and marketing is amortised across millions, or tens of millions of units sold, the per-piece cost rapidly reduces to the Variable Cost of manufacture. The <a href="http://en.wikipedia.org/wiki/Experience_curve_effects">"Learning Curve Effect"</a>, or Economies of Scale, also kicks in: for every doubling of production scale, unit costs reduce 20%-30%.<br />
<br />
Not only does the R&D capital risk get returned quickly, the cost of parts continues to drop, stimulating further demand if the market is elastic. Vendors can maintain, or increase, margins while dropping prices. It's a Golden Circle, the more you sell, the cheaper they are and the more you sell.<br />
<br />
What hasn't been widely appreciated is <a href="http://en.wikipedia.org/wiki/Moore's_law#Moore.27s_second_law">Moore's <i>Second</i> Law</a>, the Vendor corollary:<br />
<blockquote class="tr_bq">
<i>Every four years, the price of a Fabrication plant doubles.</i></blockquote>
This makes sense as well. Every new technology step is beyond current know limits and stretches existing technology - and that always comes at an increasing <i>marginal</i> cost. Each extra performance increase costs more than the last, because you always harvest the "lowest hanging fruit" first, the cheapest and easiest gains are taken before all others.<br />
<br />
This virtuous circle of faster/higher-capacity products, increased demand, cheaper prices, higher profits fuelling new R&D, new plant and higher volumes, based on PC sales, worked exceedingly well until 2008 and the Global Financial Crisis (GFC).<br />
<br />
The GFC effects, a global downturn in PC Desktop & laptop demand, was reinforced by the challenge from the smartphone and tablet markets. In Economics, they are <i>substitutes,</i> for at least part of what consumers & businesses used PC's for.<br />
<br />
Since 2010/2011 PC sales have been in decline.<br />
<br />
What hasn't washed through yet is the effect on market segments that have been dependent on the PC market for their R&D and manufacture. The same plants and technologies that produce cheap CPU's for laptops also manufacture the fastest, most expensive server CPU's.<br />
<br />
High-end CPUs might comprise 1% of the market, but they share the same plants and fabrication R&D are the cheapest, high-volume CPU's. It "only" costs $20M-$30M to design a CPU for a new "process" and nothing needs to change in the (now) fully automatic manufacturing plants.<br />
<br />
This low additional cost allows vendors like Intel to leverage their high-volume commodity production lines and profitably create and sell low-volume, high performance products. But <i>only</i> while the commodity market can pay for the whole new investment, R&D and new plant.<br />
<br />
<b>Software</b><br />
<br />
Microsoft rode the PC revolution from 1981 to 2009, increasing sales and profits every year. They traded on the underlying volume economics and Silicon Revolution to sustain performance.<br />
<br />
Describing Microsoft software created the term "bloatware", meaning it consumed radically more of every resource (CPU, RAM, Disk), or 'footprint', at every release, cancelling or negating any hardware performance gains and forcing domestic and business users to upgrade well before the hardware physically needed replacement. Since 2005, upgrade cycle have slowed.<br />
<br />
Microsoft failed as well in basic Software Engineering processes, causing the 2005 "Longhorn Reset", where around 25,000 man-years of effort was discarded, a massive financial write-off. In the marketplace, this competency & quality problem showed up in poor Security and a radically inferior User Interface.<br />
<br />
People mostly don't realise that Microsoft makes $15 in license fees on every smartphone sold, Android and iPhone, because they hold the basic patents. For nearly 10 years, starting with Windows-CE, Microsoft had the smartphone market to itself.<br />
<br />
The revolution of the Steve Jobs iPhone was not the code and technology, but the Software, Security and User Interface <i>done right. </i> The market spoke and iPhone sales soared, whilst a number of Windows Phone releases have languished in obscurity, even with the #1 mobile phone vendor, Nokia, insanely dropping all other phone operating systems, causing its business to collapse and eventually be acquired - by Microsoft.<br />
<br />
Microsoft tied it fortunes to the global PC market. With that market now collapsing due to longer "refresh" cycles and substitute products, their business model is under threat.<br />
<br />
<b>Technical Ratios and Economic Ratios have both changed in 30 years</b><br />
<br />
In 1981, the CPU at 4.77Mhz, in the IBM PC didn't need a "cache", because <i>DRAM was faster than the CPU</i>.<br />
<br />
In 1991 and the advent of the first full CMOS CPU on a chip, the Intel 486 at 25Mhz (then 50 & 100Mhz), DRAM was now slower than the CPU. Cache memory, small at first and once the preserve of super computers, became standard in consumer PC's.<br />
<br />
Ever since, the gap between CPU's and DRAM speeds has widened. Even with multi-core CPU's and their relatively static 2.5Ghz-3Ghz clock, the gap widens as more cores demand more reads/writes per second from DRAM.<br />
<br />
In the earliest value computers, some used rotating "drum" storage as main memory. The forerunner to the modern hard disk was faster than any other technology.<br />
<br />
Quickly, other faster, and more expensive per bit, technologies were developed.<br />
DRAM has continued to get faster, albeit not keeping up with CPU's, while Hard Disk Drives, HDD's, have increased quite slowly in both average access time and streaming throughput - the number of bytes/second they can transfer.<br />
<br />
As disks have had more bits stuffed onto them, two factors have changed: the number of bits-per-inch stored along a track, and the number of tracks-per-inch across the disk. The combination of the two is the "areal density", the number of bits packed into a single square inch. This has risen from a few thousand to now almost a trillion (a million million).<br />
<br />
For a given increase in areal density, half the amount comes from bit-per-inch along the track and if the disk rotates at the same rate, the transfer rate increases by that amount. Which is a great outcome, for free, your disk reads off your data that much faster.<br />
<br />
But it comes at a cost. Your 4GB disk transfers at twice the rate of your 1GB disk, but because there's now four times as much data, it takes <i>twice</i> as long to read the whole disk.<br />
<br />
The first 3.5 inch drives from Shugart, now Seagate, were 100MB and transferred at 0.6-1MB/s. In 2-3 minutes, you could read the whole drive. Current 4TB drives are 40,000 times larger, but only transfer 200 times faster, ~120MB/sec. To read your whole 4TB drive now, say for a backup, takes 10 hours - more if you're using it for other things! This blow out in "full drive scan time" hasn't garnered much attention, either.<br />
<br />
So we've had this three way gap opening up between CPU speeds, DRAM speeds and Hard Disks, together with this blowout in full drive scan time. <i>The fundamental technical ratios of all systems have changed radically, yet we persist with many of the same system designs.</i><br />
<b><br />
</b> <b>Economic Ratios</b><br />
<br />
Not only have the technical ratios changed, but also relative costs.<br />
<br />
In the mid-1980s, Businesses paid $100-500,000 (double that in today's dollars) for a minimum disk setup. The IBM 3380 stored 2.5GB, took twenty square feet of floor space and used 10kW-20kW of power.<br />
<br />
Compare this to sub-$200 for a 4,000GB 3.5 inch drive in PC that uses 8W-10W. That's a phenomenal change in cost-per-bit, Watts/bit as well as figures I haven't quoted for transfer rate and average 'latency' (time to read data).<br />
<br />
Around 1700 3.5 inch drives can be put into a single rack, surprisingly, that's a similar cost and power usage, for 3 million times the capacity (2.75PB), accessed around 2,000 times faster.<br />
<br />
While in 1988, 'average' IBM customers paid $750,000 (equivalent to $1.5M now), there are very few businesses that store even 1PB. Storage Arrays are still expensive, but because they do a lot more and squeeze out more performance than a simple "Box of Disks".<br />
<br />
In terms of average wages, Storage has come down from costing 35+ years of average wages to a month average wage. A full installation, CPU, DRAM, Disks, Tapes, Printers and Controllers, would've cost 5-8 times the price of Storage ($3-6M). Maintenance costs are typically 20%/year of purchase price. A staff of 15-20 were necessary to operate and administer a computer this size. A team of 10-20 Analysts and Programmers were also needed.<br />
<br />
The mainframe cost around 250 people's wages, was replaced in 2-3 years, cost 50-100 wages/year to operate and needed another 20-50 people to run it. On a yearly basis, an average mainframe cost 200-250 wage-equivalents per year to own and operate.<br />
<br />
For $10,000, or 15% of average annual earnings, a high-performance server, including "enough" storage can be purchased. On average, administrators look after 5-10 servers. Today, we also have Storage and Database Administrators, as well as System Admins and Operators, but many fewer. Programmers still abound. For Desktop PC's and Workgroup Fileserver, a single admin support 100-500 PC's and 5-10 file servers.<br />
<br />
While IT Departments have grown considerably in head count and budget, so has the work they perform, what they support and the "hardware fleet" under their control.<br />
<br />
For the same proportional compute capacity, a rather small server kept for 3-5 years, the full yearly cost is 25%-50% wage equivalent: <i>a 500-fold reduction to businesses</i>.<br />
<br />
In 1985, it was worth it for Admins to invest months in saving even 5% of disk space (two full years wages in CapEx, and same again in on-costs)<br />
<br />
Now, when a new drive costs one day's pay, it doesn't make sense for a business to invest much time<br />
<br />
Thirty years ago, labour was under half your computing cost and The Computer cost 100 times more a person.<br />
<br />
Today, PC's cost a couple of days wages, servers less than a month's wages and staffing costs are 60% or more of IT budgets. Additional costs, like Help Desk software, software licensing, networking and specialist training, consume a large fraction of IT budgets. Server room hardware and operations are now a relatively small fraction of Enterprise computing budgets.<br />
<br />
This is without taking into account the reason we use computers in the first place: to increase the productivity of the people in the organisation. Computers and I.T. produce a benefit to the business, or they don't get used (versus bought. Management often confuses "having" a system with it being used and being useful. Another topic entirely.)<br />
<br />
When you take the hourly turnover of Organisations supported by computers & I.T., a mid-sized business of 500 people is betting $50-$100,000/hour on everything working. System failures cost twice: once in lost work hours paid and again in opportunity cost - revenue/turnover lost during the outage.<br />
<br />
<i>In thirty years, we've gone from labour being 100 times cheaper than a computer to 100 times as expensive. We can now afford to trade computer time and resources for staff time and productivity.</i><br />
<br />
The "efficiency" of computers is now subsidiary to the productivity of staff and the efficiency of the organisation, yet Computing & I.T. still seldom make the calculation, rarely collect the necessary data and almost never justify I.T. Reliability and Performance upgrades in terms of direct Organisational savings and as Insurance against outages. I.T. and business practices both have to change to embrace the new reality.<br />
<br />
A direct consequence of 3 decades of PC's in businesses and a decade of ubiquitous Internet on the Desktop is that Enterprise I.T. services are maturing. Less new hardware is bought, fewer new systems licensed and brought into service, though conversions are more common, and both PC (desktop and laptops) and servers have had their average service life extended. We keep hardware longer because it still works well enough.<br />
<br />
In all this, more storage space is sold each year. It's hard to get good figures, but from 30%-60% increase per year is estimated. Which makes you wonder what it all gets used for. Cat pictures or real business uses?<br />
<br />
<hr />
<a href="https://www.blogger.com/blogger.g?blogID=29583699" name="1"><b>Summary</b></a><br />
<br />
I.T. Hardware and Software vendors are facing a major disruption in their business model: the express train of steadily increasing demand, underpinned by PC technologies, has stalled and is slowing.<br />
<br />
In Storage, Enterprises and consumers continue to buy more capacity, but it doesn't show up in Hard Disk sales nor vendor profits. (There's only two major manufacturers left, Seagate and Western Digital).<br />
<br />
Because the PC market has paid for the development of capacity/speed increases in CPUs, DRAM and Hard Disk (Flash is paid for with consumer electronics as well), as that market collapses, and it seems to be doing so at an increasing rate, then cash-flow dries up and financial support for new product dwindles.<br />
<br />
Vendors producing for the "mobile" market, smartphones and tablets, are still experiencing increasing demand, volumes and profits.<br />
<br />
The virtuous circle no longer operates to fund exponentially more expensive R&D and manufacturing plant. based on PC technology. With lower Return on Investment, longer payback periods and more uncertain sales volumes, vendors of all PC-based technologies will either slow R&D and new products, or cancel programs entirely.<br />
<br />
By 2015, we'll know how this goes. That could be the year that PC technology freezes, after which we'll get only incremental improvement or the very occasional "breakthrough" product.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com4tag:blogger.com,1999:blog-29583699.post-68835239192644364952014-03-18T10:51:00.000+11:002014-03-18T14:29:27.590+11:00Microsoft, IBM and the Price of SuccessMicrosoft's business is now clearly "under performing" with even the Board admitting it. Steve Ballmer has finally been pushed out. The MSFT Board could've followed IBM in 1993 and appointed an outsider, but they chose an long-term insider.<br />
<br />
The imminent "death" of Windows-XP (8-Apr-2014, free security updates end) shows, in my mind, a clear <i>misattribution</i> effect by Microsoft: they've confused cause and effect, that people bought their <i>brand</i>, not just a "standard" commodity product.<br />
<a name='more'></a><br />
<a href="http://arstechnica.com/information-technology/2014/03/weeks-before-expiration-date-windows-xp-still-has-29-os-market-share/">Windows XP, with 29% of PC O/S market share</a> vs 47% for Windows 7, is still a large product segment.<br />
By orphaning it's second largest user base, Microsoft is making two very strong statements:<br />
<br />
<ul>
<li>they aren't interested in building incrementally on success, they could easily get $5/year from Win-XP home users for updates (paid for 3-4 years in advance) OR create "son of XP" as a cheap, lightweight supported option for XP owners. They could make $25-$50 from those millions of current customers, with almost no investment by MSFT.</li>
</ul>
<ul>
<li>they believe people's primary purchase decision factor was "Microsoft" not a standard commodity platform.</li>
</ul>
<div>
How can successful companies confuse the causes of their success? These are <i>smart</i>, informed and fiercely rational <i>guys</i> (mostly still) aren't they? With a whole lot riding on their decisions: their careers and future income/reputation. <i>Skin in the game,</i> in spades. All pointers are they should be getting this right.</div>
<div>
<br /></div>
<div>
But from IBM, we know that success blinds marketers, managers, CEO's and boards. <a href="http://www-03.ibm.com/ibm/history/history/decade_1980.html">In 1981</a>, 20 years after the launch of the IBM 360 mainframe, IBM had 60% by revenue of the entire market: they were 50% bigger than <i>all</i> their competition combined. "Successful" doesn't come close to describing them.</div>
<div>
<br /></div>
<div>
The IBM PC released that year became the <i>standard</i> PC, not just in offices, but everywhere. One small factor annoyed IBM, "clone" makers had copied their design and were stealing sales.</div>
<div>
<br /></div>
<div>
IBM, in 1987, responded with a known, effective strategy: they changed the product architecture, expecting to force those many clone makers to pay license fees. The new <a href="http://en.wikipedia.org/wiki/IBM_Personal_System/2">PS/2</a> came with a new video card (<a href="http://en.wikipedia.org/wiki/IBM_Personal_System/2#VGA_video_connector">VGA</a>), new mouse & keyboard connectors, new backplane (<a href="http://en.wikipedia.org/wiki/Micro_Channel_Architecture">MCA</a> - Microchannel Architecture), new proprietary networking (<a href="http://en.wikipedia.org/wiki/Token_ring">Token Ring</a>) and a new, much better Operating System (<a href="http://en.wikipedia.org/wiki/OS/2">OS/2</a>). Even new and better connectors to disk drives were added: a single connector for both signals and power.</div>
<div>
<br /></div>
<div>
The <i>plan</i> was new hardware & software together, but OS/2 was late and they shipped PS/2's with MS-DOS, undermining the strategy and costing sales along with market share.<br />
<br /></div>
<div>
IBM, like Hewlett-Packard, design and build <i>great</i> hardware. The PS/2 would've been a delight to use and great economics for large enterprises needing reliable PC "fleets". But great hardware takes time to design, build <i>and</i> test. Smaller, more agile hardware vendors responded in months, not the 3 or more years it had taken IBM to roll out the PS/2.</div>
<div>
<br /></div>
<div>
Only, customers didn't follow the IBM Marketing's script. Vendors kept building "ISA" (Industry Standard Architecture) PC's, not MCA, but upgraded them to use the PS/2 keyboard/mouse connector & interface and VGA. Ethernet won the networking wars and it was another 15 years before we saw laptops with single connectors for disks.</div>
<div>
<br /></div>
<div>
<a href="http://en.wikipedia.org/wiki/History_of_IBM">By 1993, IBM was haemorrhaging money</a>: $16 billion in losses over 3 years, half that in 1993 alone. Things had gone <i>bad</i> for them, very quickly. Louis V. Gerstner was brought in from outside IBM and didn't just stop the decline, but put them back into profitability. He stayed 10 years, leaving in 2002. </div>
<div>
<br /></div>
<div>
How could a company of 350,000 of the best computing and marketing people on the planet, led by seasoned professional managers with impressive records, get it <i>so</i> wrong?</div>
<div>
<br /></div>
<div>
My thesis is that successful companies <i>must</i> get disruptive decisions wrong - our minds are wired that way...</div>
<div>
<br /></div>
<div>
It isn't just <a href="http://en.wikipedia.org/wiki/Confirmation_bias">confirmation bias</a>, <a href="http://en.wikipedia.org/wiki/Groupthink">group-think</a>, <a href="http://en.wikipedia.org/wiki/Self-deception">self-deception</a> or <a href="http://en.wikipedia.org/wiki/List_of_cognitive_biases">perceptual bias/filtering</a>, it's a basic misattribution effect:<i> at some point successful enterprises (not just business) confuse Cause and Effect.</i></div>
<div>
<br /></div>
<div>
The very act of being successful in the medium-term renders the leaders and administration of an enterprise, be it business, military, political, religious, or academic, to believe their success is more than luck or serendipity. We are now seeing aircraft pilots crash because they believe their mental model "the automation/instruments are <i>always</i> correct" and discount the evidence of their senses.</div>
<div>
<br /></div>
<div>
Our brains are wired to make sense of the world and look for causality. We form very strong, powerful mental models that generally work for us and, when correct, create successful outcomes for us. This cognitive ability has allowed us to survive almost anywhere, alter the large-scale environment to suit our needs and create everything around you.</div>
<div>
<br /></div>
<div>
But it comes at a cost. Sometimes we confuse Cause and Effect. We can see it in other times & cultures and their predisposition to believe in magic or superstitions. For those people and cultures, this was a rational, evidence-based belief-system. <i>Everything</i> they observed confirmed their mental models of how the world works.</div>
<div>
<br /></div>
<div>
So to IBM and Microsoft and thousands of successful enterprises before them that stumble and fall. It took the Roman Empire a thousand years to implode... But fall it did.</div>
<div>
<br /></div>
<div>
You <i>only</i> get to find out your mental models are incorrect, which are very refined and have served you long and well, after a resounding collapse. It's not obvious to anyone when they start failing because "that's just a natural variation" or "we're facing challenges, but we've overcome obstacles before".</div>
<div>
<br /></div>
<div>
It took IBM around 5 years from the release of the ill-fated PS/2 to fail more massively than anybody could anticipate. How long will it take Microsoft to hit the ropes after abandoning 30% of their user base?</div>
<div>
<br /></div>
<div>
Ordinary people bought Windows-XP because "it was there" and "its the standard".</div>
<div>
Hardware vendors provided Windows-XP because there was consumer demand and no affordable alternatives.</div>
<div>
ISV's, Independent Software Vendors, wrote software for Windows-XP because it was the dominant Operating System. Serving that market alone was a correct strategic decision.</div>
<div>
<br /></div>
<div>
Being successful in the market created more success. Higher PC sales were caused by other factors, like falling hardware prices, aggressive competition amongst vendors and the Internet revolution. All of a sudden, PC's were a lot more valuable <i>networked</i> than standalone. Microsoft benefitted by being in the right place at the right time with a good enough product. They beat out all the early competition because they took some massive bets that paid off. We might all be running one of a dozen different O/S's if their bets had failed.</div>
<div>
<br /></div>
<div>
Microsoft, and Ballmer in particular, never recognised this. Ballmer and the Board, in my view, believed something <i>they</i> did led directly to their outrageous success. The very obvious contradictory evidence that they'd been unable to replicate their success, despite tens of billions spent trying, was simply ignored. The massive wake-up call that their engineering processes were deeply flawed, the 2004/5 "Longhorn Reset" - a 30,000 man-year "FAIL", was also ignored.</div>
<div>
<br /></div>
<div>
Humans are amazingly good at ignoring disconfirming evidence: IBM and Microsoft executives aren't unique or special.</div>
<div>
<br /></div>
We have absolute proof that a sizeable fraction of the population would rather die than change: <br />
<blockquote class="tr_bq">
smokers continue to endanger their lives in spite of every warning while healthy, and an alarming proportion continue smoking even when diagnosed early enough to do something.</blockquote>
Ballmer and his board, like IBM's many CEO's and board, aren't evil, mad or bad in missing the obvious. They're just human with the same frailties as the rest of us and countless leaders before them.<br />
<br />
The trick would seem to be having someone powerful around that will tell you "It ain't so".<br />
My guess is that Bill Gates saw the writing on the wall and after his view was rejected, stood aside. But we'll never know because he'll never say.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-2831694241082740672014-01-27T13:04:00.000+11:002014-03-18T09:12:26.871+11:00Security: Healthcare, Computers and Ignorance/Inaction.<a href="http://stevej-on-it.blogspot.com/2013/01/security-computer-security-for-business.html">A year ago I wrote</a> that with the then epidemic of "ransomware" attacks the Hackers had learned how to monetise remote attacks on Healthcare practices. That piece included detailed suggestions on minimum necessary practices and questions for suppliers and vendors.<br />
<a name='more'></a><br />
<br />
This year started with <a href="http://www.abc.net.au/news/2014-01-16/australian-pharmacies-targeted-in-ransomware-attacks/5203970">the Pharmacy Board, the professions' registration/regulating authority, taking the unprecedented step of writing to <i>all</i> pharmacies</a>, warning them of the danger to patient safety that "ransomware" now presents. I assert that for a well managed business, "ransomware" should be no more trouble than a failed disk.<br />
<br />
This is a legal watershed moment: patients who suffer harm or loss because of a ransomware event now have cause for a negligence or malpractice action against the business, its principals, its I.T. staff/support company and the software vendor. Both Pharmacy and Medical Practice software is dominated by a single vendor, multiplying many fold the risk of compromise of clients.<br />
<br />
This near monopoly places a high duty of care on the vendor, as the prime, even sole, source of I.T. Security advice to the businesses, to both adequately inform them of the full risks involved <i>and</i> confirm all their clients have adequate remediation and recovery, not prevention, processes in place.<br />
<br />
The key to surviving <i>any</i> business disaster is "Prior Preparation and Planning", as our military friends would say. Computing and I.T. are just <i>one</i> Point of Failure: the rigorous planning and practice notionally already in place in every practice for fire, flood, equipment failure, major theft or malicious damage <i>should</i> cover loss of critical business data and patient records.<br />
<br />
Remote attacks launched over the Internet have built steadily in volume and sophisticated since <a href="http://stevej-on-it.blogspot.com/2007/04/end-of-internet-or-microsoft-users-net.html">"the Hackers turned Pro" at the end of 2004</a>. Coming up to the tenth anniversary of professional criminal activities over the Net, there are <i>no</i> grounds for any healthcare professional, owner or company director to claim ignorance.<br />
<br />
We've seen nation states enter the field with both <a href="http://en.wikipedia.org/wiki/Stuxnet">Stuxnet</a> and <a href="http://intelreport.mandiant.com/">Mandiant's APT1 report</a>. We must now presume every developed economy runs an active "Cyber Command" pursuing their interests over the Net as an extension of their military, intelligence and business espionage activities. "Cyber Command" and SigInt players behave identically to "Special Forces": they avoid detection, have deep resources and act with supreme patience. They also identify critical weaknesses and create highly targeted attacks against them. They will deliberately create mischief and mayhem, even widescale disruptions, to cover covert operations.<br />
<br />
Since the Snowden revelations we, the public, know that the capabilities and actions of National Security players are much higher and broader than ever guessed. <i>The Internet Security threat is much worse than you can imagine.</i> If you are responsible for Healthcare records, not acting to protect against these high-level threats is no longer a legally defensible position. Ignorance and Inaction aren't an option now.<br />
<br />
The problem with all these very sophisticated Cyberwarfare tools being released into the wild is they go into the wild. They <i>all</i> get picked up and reused by hackers, scammers and professional criminals. All software & network tools and techniques available to US Cyber-Command become available to hackers and criminal gangs as they are released into the wild. It only takes 12-24 months to see them in common use elsewhere because there are only a very few suppliers of malicious <i>code</i> and they're very good at what they do.<br />
<br />
There are already an increasing number of business failures linked to failed I.T. systems, with some being linked directly to security breaches.<br />
<br />
The most public business failure in Australia caused by a security breach was <a href="http://www.abc.net.au/news/2013-05-20/cyber-crime-wipes-out-millions-of-dollars-and-a/4701836">Distribute.IT in 2011</a>. Nobody was ever charged over this because all evidence was removed when the disks were wiped and <i>the backups had been turned off for an extended period beforehand</i>.<br />
<br />
What's your take-away from this? A criminal's would be "Wow! A blueprint to escape detection."<br />
We know from the last 10 years of criminal hacking on the Net that they learn, adapt and overcome. This action/outcome will not have gone unnoticed.<br />
<br />
The business directly lost $5 million, <a href="http://www.abc.net.au/technology/articles/2011/06/22/3250581.htm">but the disruption and non recoverable losses suffered</a> by the 4,800 clients was never tallied. There was no point in their launching any legal action, the business and its owners were wiped out financially, there was nothing avaiable to pay any compensation.<br />
<br />
And one last "little complication": on Tuesday 8th April, 2014, the <i>last</i> public security updates will be released for Windows XP, the most popular Operating System from Microsoft first sold in 2001.<br />
<br />
Already in this blog I've predicted that within 12 months <a href="http://stevej-on-it.blogspot.com/2014/01/now-read-this-why-munich-open-source.html">no Windows XP system connected to the Internet will be free of malware</a>. The hackers can read calendars too and will be saving their efforts and munitions, especially "<a href="http://en.wikipedia.org/wiki/Zero-day_attack">Zero Day exploits</a>" (previously unknown and unpatched backdoors), for when they can operate freely.<br />
<br />
We know from Mandiant that in the best exploits, the victim <i>never</i> knows, unless the attacker wants something from them.<br />
<br />
So there's the thing:<br />
<ul>
<li>Do your backups, <i>religiously</i>.</li>
<li>Test your backups, <i>frequently</i>.</li>
<li>Don't <i>ever</i> think "I'm small fry, who could be interested in me?"</li>
<ul>
<li><i>Expect</i> to get compromised.</li>
<li><i>Be ready</i> to recover from multiple worst-case scenarios.</li>
</ul>
</ul>
There are many eyes watching your system(s) <i>now</i> and they're actively looking for ways in, <i>if they aren't already.</i> Indian call centres have been actively trawling phones in my area for the last 2 weeks attempting various "<a href="http://en.wikipedia.org/wiki/Social_engineering_(security)">Social Engineering</a>" attacks. These activities and "<a href="http://en.wikipedia.org/wiki/Phishing#List_of_phishing_techniques">spear phishing</a>" guarantee that someone in your business <i>will</i> open up your systems to attackers or give away the passwords. Mandiant reports these were some of the many techniques used to silently take over <i>thousands</i> of business computers throughout the world.<br />
<br />
Cyber Security only ever ramps up in sophistication and intensity: <i>you can't imagine the threat landscape in 12 months, let alone 5 years. </i>The Internet is currently a much more dangerous place than you know, in a year that danger will increase, in 5 years it will be beyond recognition.
steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-84987212214154506932014-01-07T09:29:00.001+11:002014-03-18T08:31:15.428+11:00Now Read This: Why the Munich Open Source Conversion won't be replicated in Australia.Whichever side of the Open Source vs Proprietary Software debate you lie on, this article is a "must read". The headline take-away is: "Our goal was 'Freedom', to become independent."<br />
<blockquote class="tr_bq">
<a href="http://www.techrepublic.com/article/how-munich-rejected-steve-ballmer-and-kicked-microsoft-out-of-the-city/">How Munich rejected Steve Ballmer and kicked Microsoft out of the city</a>, <a href="http://www.techrepublic.com/meet-the-team/uk/nick.heath/">Steve Heath</a>, 18th Nov, 2013.<br />
<a name='more'></a></blockquote>
<ul>
<li>This piece proves it's <i>possible</i> for an organisation to leave the Microsoft World.</li>
</ul>
<ul>
<li>They couldn't migrate <i>everything.</i></li>
<ul>
<li>They prioritised what they'd migrate or replace and adopted different solutions suitable for each application.</li>
<ul>
<li>This was very similar to dealing with Y2K.</li>
</ul>
<li> Solutions like VM's & Virtual Desktop Environments are needed.</li>
</ul>
</ul>
<ul>
<li> As Microsoft says, Free Software isn't <i>Zero Cost</i>, but <i>is</i> cheaper.</li>
<ul>
<li>what <i>nobody</i> discusses is how much staying on the "Microsoft Forced Migration Path" costs. This is the real baseline for comparison.</li>
<li>They'd have been up for the same, or higher, costs migrating from Windows XP to a Windows 7 or 8 (Vista, the least popular O/S ever, was never an option).</li>
</ul>
</ul>
<ul>
<li>Munich started planning and preparation in 2002/3, rolled out a major Application, Open Office, soon after and then in 2012 upgraded all desktops to LiMux over 12-18 months:</li>
<ul>
<li>Sorting out the "IT zoo" was expensive & necessary, even if they'd stayed with Microsoft.</li>
<li>They brought the organisation into the modern era of the "Standard Operating Environment" - a single, secure build used on all Corporate Desktop machines.</li>
</ul>
</ul>
<ul>
<li>As more organisations embrace Open Source it, it's cheaper and easier, and suitable staff and services more widely available.</li>
</ul>
<ul>
<li>Munich created their own "LiMux", based on Ubuntu.</li>
<ul>
<li>a team of 25 develop, roll out and support LiMux.</li>
<li>fewer than are needed to support the Windows PC's.</li>
</ul>
</ul>
Heath writes near the end that Organisations who've have failed in their Open Source projects were only doing it to save money, the implication is this as a sole motivator is a guaranteed "fail". Like any large disruptive change, top management support and active Change Management are necessary.<br />
<br />
With the looming deadline, April 8th, 2014, for the last "patch Tuesday" security upgrade of Windows XP, a lot of older desktops will be forcibly orphaned or discarded. Upgrading these old PC's to Windows 7 or 8 is impossible or uneconomic.<br />
<br />
Users who wish to keep their old PC hardware running <i>must</i> convert to some flavour of Linux.<br />
Users who buy new hardware get NO advantage by sticking with Windows: Microsoft gain no benefit from being the existing Operating System. Converting to Apple is likely to be a much cheaper and easier option than going to Windows 8 for XP users.<br />
<br />
<b>Doing nothing is not an option:</b> there are enough "Zero Day exploits" <i>on record,</i> geek-speak for "security bugs with NO published fix", that it's guaranteed within 12 months all Windows XP machines connected to the Internet <i>will</i> be "owned". No sane owner would put themselves in this position.<br />
<br />
I told an old friend in a Government Agency, a long time advocate for Open Source, that <i>Security</i> should be an overwhelming reason for the Public Service to aggressively pursue an Open Source strategy:<br />
<blockquote class="tr_bq">
<i>If you don't have the source to your Operating System, Tools and Applications, and the people to understand & fix the code, then why would you think you can </i>ever<i> be 'secure'?</i></blockquote>
<ul>
<li>After Stuxnet and its <i>two</i> successors, we now <i>know</i> that "Air Gaps" are not nearly the impenetrable barriers for Secure Networks they once were.</li>
<ul>
<li>The Iranian centrifuge controllers attacked & compromised were on an isolated network, yet were taken over and without being detected.</li>
</ul>
</ul>
<ul>
<li>After Mandiant's APT-1 (Advanced Persistent Threat, #1) revelations, we now know that there are many Nation-states with the will, capability and motivation to run many military-grade, very long-term cyber-intelligence missions.</li>
</ul>
<ul>
<li>If the USA, and possibly Israel, can access highly protected environments, then so can other actors, state or commercial/criminal. The code is out there in the wild and can be reverse engineered and retargeted for other uses.</li>
<ul>
<li>The number of <i>Zero Da</i>y compromises available to both for-profit hackers and "Cyber-Commands" is unknown, but suspected to be high.</li>
</ul>
</ul>
There are now <i>two</i> critical commercial risks for every Organisation or Individual that's tied itself to the Microsoft Ecosystem. Remember that Microsoft is under NO obligation to continue any service or product, or maintain current pricing & availability:<br />
<ul>
<li>Microsoft may ditch on short notice one or more products and services that are <i>mission critical</i> for the business.</li>
<ul>
<li>In this new world of 'cloud' licensing, the time from the announcement to your software not running, could be as short as <i>one</i> month. Even with a decade warning and preparation, Y2K turned into a mad scramble. This scenario will be many times worse.</li>
<li>Microsoft have a history of suddenly dropping products and orphaning users without support or migration path. </li>
</ul>
</ul>
<ul>
<li>A Microsoft under New Leadership <i>will</i> attempt to redefine itself and <i>will</i> shed "non-strategic" assets and products.</li>
<ul>
<li>This may take the form of selling all, or part, of the Business and Services divisions to End of Life Consolidators, like Computer Associates and Fujitsu.</li>
<ul>
<li>Anyone with a mainframe who's been trapped this way will tell you its not a happy experience, nor cheap.</li>
<li>Consolidators pay a premium to acquire End of Life products, then exploit the dependence of customers to create very handsome profits for themselves. </li>
<li>Maintenance and upgrades are minimal, <i>if</i> they happen at all.</li>
</ul>
</ul>
</ul>
There's a few simple questions that Senior Management, especially in the Public Service, need to be asking about these risks to their business from Microsoft changing it product lineup and charges. If you thought Y2K was expensive and disruptive, you haven't seen <i>anything</i> yet:<br />
<ul>
<li>How much more dependent is your business, especially routine front-office operations, on computers and I.T. compared to Y2K, 15 years ago?</li>
</ul>
<ul>
<li>Have you contingency plans written, let alone tested, for any of your Microsoft products being suddenly withdrawn?</li>
</ul>
<ul>
<li>What level of dependancy do you have on the Microsoft Ecosystem?</li>
<ul>
<li>Could your business even operate <i>one day</i> in "manual mode"?</li>
<li>How quickly could you cobble together replacements if every other major Corporate customer in Australia was competing for the same resources and services?</li>
<ul>
<li>Such as during Y2K and GST projects in 1999/2000.</li>
</ul>
<li>If you contingency plan is "get Microsoft to help us", that will fail because they won't be able to cope with demand, even <i>if</i> they offer help and support.</li>
</ul>
</ul>
CEO's, CFO's and CIO's who haven't already answered all of these questions today are likely to have a very rude awakening, the least of which will be losing their jobs.<br />
<br />
We know from the collapse of IBM's business in 1992 and 1993, <i>still</i> the largest corporate losses survived by a US company, that problems <i>do</i> arise for "too big fail" I.T. vendors and can take the market and customers completely by surprise.<br />
<br />
IBM is far from the only leading I.T. vendor to fail: Unisys, the 1986 merger of numbers #2 and #3 (Sperry Univac and Burroughs), led to a dysfunctional, unfocussed, strife ridden company that imploded and was abandoned by 90% of their customers in only a few years.<br />
<br />
The major lesson from IBM is the frightening speed with which these reversals occur: within 6 months of the first warning signs, the business was uncontrollably bleeding red-ink. The causes of sudden collapse mean <i>there is no easy way back to profitability</i>. IBM embraced Open Source, added Services and PC's as its major product lines and reorganised its mainframe business, radically reducing prices and taking a massive, though temporary, hit to their bottom line.<br />
<br />
Microsoft has only <i>two</i> profitable software products, and they go back 20-30 years:<br />
<blockquote class="tr_bq">
Operating Systems and Office Applications.</blockquote>
What's salutary for both investors and customers, is that, unlike IBM before it, Microsoft has <i>never</i> created another substantial line of business, despite repeated attempts and tens of billions of dollars wasted. They <i>owned</i> the mobile and smartphone business from 1995-2005, yet were slammed by the iPhone and became irrelevant in the marketplace within 24 months. They <i>invented</i> "Tablets" but have yet to release a competitive product.<br />
<br />
Microsoft have attempted to enter <i>every</i> market, business and consumer, that Google, Apple, Amazon and VMware have shown are profitable and in high-demand. Without fail, Microsoft has failed to create another significant line of business for itself. The X-box has been a marketing triumph and financial failure. It only returned its first profits in the last few years and in terms of total profit, is a footnote at best to the business.<br />
<br />
These problems were apparent in 2005 when Microsoft announced they'd abandoned around 25,000 man-years of effort in 2004, with the "Longhorn Reset". Their corporate Project Management and Software Engineering skills, core competencies for I.T. vendors, were so throughly compromised, they were unable to achieve the basics: bring a product to market.<br />
<br />
Windows XP, in 2002, was the last successful and widely welcomed Operating System developed by Microsoft. Since Vista, Microsoft Operating Systems have only been grudgingly taken up by users, especially by Corporate users. The 8th April 2014 "End of Life" for XP may prove to be a turning point in the fortunes of Microsoft, as they discontinue their most popular and reliable O/S product.<br />
<br />
Microsoft failed to master what ever other successful hardware company does: yearly system updates.<br />
Since Windows NT 1.0, around 1993, they have failed at this basic & essential task.<br />
Look at <i>every</i> other large commercial Operating System vendor: as a matter of course, they have a yearly release cycle, sometimes with major releases every 2-3 years.<br />
<br />
Well run Operating Systems groups are able to roll out tested, robust "production ready" releases <i>every</i> year and reduce security issues with every release.<br />
<br />
Microsoft has shown <i>for twenty years</i> that they cannot meet this minimum, necessary standard of Software Engineering for Operating Systems. This is not just a failure of technical and project management staff, but of management to the highest levels.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-39024162162558944262013-08-14T11:08:00.000+10:002013-08-14T11:08:08.720+10:00Microsoft: End Games have started.<a href="http://www.pcworld.com/article/2046556/microsofts-surface-rt-snafu-prompts-shareholder-suit.html">News report on a shareholder class action against the Microsoft management team </a>and failure of "Surface", and consequent $900MM writedown of inventory.<br />
<br />
My reaction to the friend who sent me the link:<br />
<a name='more'></a><br />
<br />
Ballmer is acting like an Emperor...<br />
<br />
That was OK for Bill to do.<br />
<ul>
<li>he had the full commitment of the troops</li>
<li>was technically competent (though he gets a 'fail' on good Software Engineering practices. Think the Security initiative and Longhorn Reset. All avoidable & obvious.)</li>
<li>and a brilliant marketer, prepared to change with the market.<br />Though Paul Allen was the one with the insight and vision.</li>
</ul>
<br />
All of which meant MSFT could "turn on a dime", as they did with the "<a href="http://www.wired.com/thisdayintech/2010/05/0526bill-gates-internet-memo/all/">internet Tidal Wave memo</a>" (1995).<br />
<br />
Microsoft under Ballmer is the same bureaucratic, entitled, rigid and ossified organisation that IBM was in 1981. Actually, much worse.<br />
<br />
IBM could satisfy business needs and had a very wide range of products.<br />
Microsoft is at best a "two-trick pony", (Winders + Office), which might be really be a "one-trick pony".<br />
<br />
With a one-time war-chest of $50B-$100B, how could they NOT find and fund 100 or 1,000 high-growth disruptive start-ups?<br />
<br />
Oh, that's right, it's all about Ballmer and his ego: "My Sales are Bigger than Your Sales", no <i>disruption</i> allowed!<br />
<br />
That's wonderful news!steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-4051404672999793622013-06-25T17:28:00.000+10:002013-06-27T17:38:55.837+10:00I.T. is NOT a Profession. There are NO consequences for Failure, It's Unprofessional to not Learn.A response <a href="http://delimiter.com.au/2013/06/25/yet-another-disastrous-vic-govt-it-project/#comment-614268">to a piece on Delimiter</a> reporting QLD and VIC government project failures.<br />
<br />
Compare the local IT failures with <a href="http://www.infoq.com/articles/contract-model-failure">these comments from Infoq.</a> <i><a href="http://www.flexiblecontracts.com/">Author site</a>.</i><br />
<blockquote class="tr_bq">
This is best illustrated by the findings from the US Department of Defense (the DoD).[10]<br />The DoD analysed the results of its software spending, totalling an eye-watering $35.7 billion, during 1995.<br />They found that only 2 per cent (2%) of the software was able to be used as delivered.<br />The vast majority, 75 per cent, of the software was either never used or was cancelled prior to delivery.<br />The remaining 23 per cent of the software was only used following modification.<br />That would suggest that the DoD actually only received business value from $0.75 billion of its expenditure – nearly $35 billion of its expenditure did not result in software that delivered any immediate business value.<br />[10] The results of the study were presented at the 5th Annual Joint Aerospace Weapons Systems Support, Sensors, and Simulation Symposium (JAWS S3) in 1999.</blockquote>
<a name='more'></a><br />
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
Others have commented, my responses alone are in this piece.<br />
Topics raised:<br />
- systemic problems with Project Management: projects are much more than "mere technical problems".<br />
- not just the Public sector<br />
- no difference between outcomes & work in Public and Private Sector<br />
- just not visible in Private sector.<br />
<br />
There were also a few wonderful comments along the lines of spin & misdirection:<br />
"there are no problems, you're just missing the central point, it's all wondrous"<br />
<br />
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
Are they inventing NEW ways to stuff up?<br />
Is current technology not able to meet the specs?<br />
<br />
If neither, then:<br />
- where are the Audit Office investigations, like Bureau of Air Safety investigations?<br />
- where are the *penalties* for individs, mgrs and companies for repeating Known Errors, Faults, Failures?<br />
- where are the consequences for Public Servants, PM's and line Mgrs, for allowing stuff-ups?<br />
<br />
After 60+ years of IT foul-ups, why aren't those responsible for the "Checks and Balances" _preventing_ them or ensuring "It won't happen again"???<br />
<br />
Not just an IT and management failure, but systemic failure of auditors, investigators, political process and compliance enforcers:<br />
FAILS Govt own rules for bureaucrats - "efficient, effective, ethical" use of Govt money and resource.<br />
<br />
Software and Systems are "hard".<br />
But that's NO reason to repeat or allow stuff-ups that are predictable and preventable.<br />
<br />
Can good software/systems be written: Ask NASA, ask Airbus & Boeing.<br />
You can *bet your life* it's not just possible, but practicable.<br />
<br />
Where is the outrage _and_ action (investigation, censure, retraining) by the Professional Bodies responsible and by those responsible for training Computing, Software and IT professionals?<br />
<br />
Could you imagine _any_ of the Engineering Disciplines NOT reacting to on-going Professional Failures of this magnitude and importance? I can't.<br />
<br />
Can you imagine _any_ Government Agency running without IT systems these days?<br />
They are _critically_ dependent on their IT systems, running and being correct, for daily operations.<br />
<br />
What bit of "we can allow IT failures" is Professional or Ethical?<br />
<br />
<hr />
<br />
@<a href="http://delimiter.com.au/2013/06/25/yet-another-disastrous-vic-govt-it-project/#comment-614269">Glen</a> (speaks of "fatcats" and other sources of waste and over-runs)<br />
<br />
Not fatcat, demonstrably _incompetent_ and negligent.<br />
<br />
Here are the basic management and project questions:<br />
<br />
- If we are doing this project "for a business benefit" in the same way we invest in marketing, how will we measure its _specific_ business benefits over the _full_ life of the project?<br />
<br />
- How hard is the project, how long will it take _our_ people in _our_ environment to do it?<br />
- this presumes a database of local IT projects completed, estimates and actual.<br />
- and a database of the _productivity_ of each individual, team and Project Manager.<br />
<br />
- What are the specific capabilities, both task area and degree-of-difficulty, of each IT worker that will be involved in the project?<br />
- Do we have critical shortfalls on particular task areas or in high-degree-of-difficulty tasks?<br />
(Do we have the folks we need to do the hardest bits properly?)<br />
<br />
- Have we done any projects before with similar or overlapping specs?<br />
- what do we need to repeat prior SUCCESSes?<br />
- what do we need to do to avoid previous FAILURES?<br />
<br />
These seem to me these are basic and necessary questions that should be answerable in any business whose operations are dependant on its IT & Systems and have been doing the work for more than a decade.<br />
<br />
<hr />
<br />
Glen,<br />
<br />
You've accurately stated most of the problems with IT projects and its Project Management.<br />
<br />
These have all been researched and solved many times over.<br />
At least once in each decade from 1970 onwards.<br />
<br />
Agile, XP and "Design Thinking" are just the latest go-round.<br />
<br />
The problem isn't the finding solutions, but remembering them and getting them used by this generation and the next.<br />
<br />
This is a failure of IT management, the Profession/Professional Bodies and Education.<br />
<br />
It is also a gross failure of line management - allowing IT projects to keep stuffing up.<br />
<br />
Everyone is allowed a first mistake, but after the first big or small IT disaster, an engaged and competent manager would ask two questions:<br />
- why won't it be different _this_ time?<br />
- what have you changed so that it _will_ be different?<br />
<br />
Line-management and senior mgrs love to assign blame and berate IT as incompetent and worse.<br />
But they are the people who've abrogated their responsibility and not just allowed, but encouraged, on-going IT disasters.<br />
<br />
Ask the parent of any 3-yo: what matters most, what you _say_ or what you _do_?<br />
<br />
At some point, management have to take responsibility for their lack of oversight and process improvement.<br />
<br />
<br />
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
</div>
<br />
<br />
Confirm.<br />
<br />
NO differences between Public & Private sector.<br />
<br />
Same Consultants,<br />
same Firms,<br />
same Project Managers<br />
same technical people.<br />
<br />
‘Same’ doesn’t mean ‘like’, it means ‘identical’ as in the same individuals cycle through all the big firms and work for public/private.<br />
<br />
<br />
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br />
<br />
I’ve travelled enough and set out to meet IT practitioners from all over.<br />
My evidence is this failure of IT as a Profession is the norm across _all_ the developed world.<br />
<br />
Simple proof:<br />
<br />
- Y2K had universal impact.<br />
- as does malware.<br />
<br />
<br />
<br />
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<br />
Agree.<br />
<br />
Did Westpac get roasted for throwing away $500MM-$1000MM on "CS90"?<br />
Do the shareholders _know_ anything changed?<br />
Did any of the people responsible suffer consequences?<br />
Is CS90 used as a local example in Uni courses on Project Management?<br />
<br />
*No* to all...<br />
<br />
On Standish and their CHAOS report.<br />
- created a lot of interest and controversy when it came out<br />
-while they skipped one early year, Jim Johnson and his team have continued to release new survey data every year since, allowing change tracking.<br />
- despite the apparently calamitous state of the Industry _nobody_ has replicated the series<br />
- even so, the veracity & methodology of CHAOS has been roundly criticised and attacked in the literature.<br />
<br />
What's wrong with this picture? everything...<br />
<br />
Those who say "it couldn't be _that_ bad" are either uninformed or disingenuous.<br />
<br />
The thing is, there is credible, tested research in IT, going back to the early _1970's_ that says, not only is it _less_ work to "Do it Once, Do it Right", it's also quicker and cheaper.<br />
<br />
I cannot understand why the _best_ work isn't picked up, while the industry still endlessly chases the newest fad & fashion - with unproven or questionable results. It's upside-down.<br />
<br />
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br />
[responding to "you're talking about Lean and "validated learning" "]<br />
<br />
<br />
Tom,<br />
<br />
You're exactly right, but a don't show good knowledge of historical efforts.<br />
<br />
This stuff is a) well-known and b) is shown to work and c) its always been _easier_ and _cheaper_ to do it right.<br />
Von Neuman got it right and created a high-performance, high-quality culture, but this was lost. Why?<br />
<br />
Your current understanding and language are _correct_ and I applaud you & your teams if you use them.<br />
But they are restatement and rediscovery of previous work going back at least 4 decades.<br />
<br />
My on-going points are twofold:<br />
<br />
- since early 1970's how Software can be done "better/cheaper/faster" has been studied/researched and good techniques discovered and widely disseminated,<br />
<br />
- BUT the known, effective approaches are not widely adopted nor do known "best practices" get taken up AND made stick.<br />
<br />
What's going on is NOT a Profession in action, only a disorganised "Industry".<br />
<br />
And the deeper questions:<br />
<br />
- Since the 1968 NATO Conference on Software Engineering, is there _any_ reason for Universities NOT to turn out graduates who aren't trained and experienced in _known Best Practices_?<br />
<br />
- For the last 40 years, is there _any_ reason that Management has allowed I.T. project to fail from Known Causes? It's OK for Professionals to make mistakes, but _never_ to repeat them.<br />
<br />
- As Government and Large Business have become increasingly dependent on Computing & ICT, is there _any_ reason that their "Checks and Balances" processes & groups _haven't_ starting collecting basic project & outcome data or started investigating failures and imposing "consequences" on individuals and organisations repeating, or allowing, known errors, faults and failures?<br />
<br />
I appreciate your Professionalism, insight and understanding and that you've dedicated yourself to Software Improvement - you should be applauded & recognised for that.<br />
<br />
I trying to draw attention to something much larger that's going on and that it's been noticed and _documented_ as a massive problem for around 50 years...<br />
<br />
As Larry Constantine observed in 1994:<br />
There is NO Software Crisis. Crises are time-limited, this has been going for _decades_, it's a slow train-wreck.<br />
<br />
There is a systemic problem that keeps recreating the same poor cultures, same worst-possible management and same predicable, preventable and unavoidable failures.<br />
<br />
This has all been said before:<br />
so what aren't the people that can _fix_ things, the business owners and govt ministers, taking notice?</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<br />
<br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<hr />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
</div>
steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-62726495457870881852013-06-04T14:29:00.005+10:002013-06-08T11:26:24.274+10:00Microsoft: Have missed the boat on catching up to Apple & Google.In early 2007 I started opining that <a href="http://stevej-on-it.blogspot.com.au/2007/02/microsoft-hits-financial-crisis-around.html">Microsoft would "hit a financial pothole"</a> around 2010. <a href="http://stevej-on-it.blogspot.com.au/search/label/microsoft">Thirty posts followed</a> before I gave it up: the mainstream & financial press were looking seriously at the topic. [A <a href="http://www.thephoenixprinciple.com/blog/2013/01/sell-microsoft-now-game-over-ballmer-loses.html">Forbes columnist "called the game"</a> in January.]<br />
<br />
What prompted my view in late 2006, pre-iPhone, I'd noticed that Microsoft hadn't been able to maintain it's growth above 10%. They (legally) manipulated their figures to remove extreme volatility. Since the early 1970's, I've seen a bunch of Tech companies falter, stumble and fail. Always starting with one "off" result. When the iPhone appeared, I knew the mechanism of their "pothole".<br />
<br />
I missed the GFC in 2008 and its effect (revenue down) and in 2012 went on record saying <a href="http://stevej-on-it.blogspot.com.au/2012/02/microsoft-troubles-xv-my-predition-not.html">"I missed the date"</a>. I'm a technical person, not a financial analyst.<br />
<a name='more'></a><br />
Note I didn't predict their <i>failure</i> or <i>collapse</i>, only a "pothole". How the Microsoft management respond now is what will determine the future of the company. Will they do "more of the same" or look to real change and innovative responses? That's up to the Board.<br />
<br />
My view is that Windows has a "lock" on the Enterprise Desktop, nothing touches it for 1,000-100,000 desktops. We can expect to see "Windows" still being sold & supported for at least 25 years. The fate of the company is not nearly as clear cut.<br />
<br />
Paul Wallbank <a href="http://paulwallbank.com/2013/06/04/steve-ballmers-big-platform-change/">wrote an insightful piece today</a> on the latest step in the Microsoft saga, ending with:<br />
<blockquote class="tr_bq">
The numbers aren’t lying for Microsoft. That’s why Steve Ballmer has to move fast and think creatively about the company’s future.</blockquote>
The time to address the iPhone/mobile device revolution was in 2007, <i>just like Google did</i>. Microsoft has attempted multiple "iPhone killer" releases of its products since 2008. With all its resources & expertise, how did Microsoft let Google deliver an iOS <i>competitor</i>, Android, that's gone onto become the most popular mobile O/S? This despite Microsoft <i>owning</i> the Desktop and <i>inventing</i> the smartphone and owning that market for around a decade.<br />
<br />
<i>How was that allowed to happen? Why wasn't the senior management cleared out in 2009?</i><br />
<br />
Microsoft's market dominance was accidental and <i>solitary</i>: they never managed, despite many attempts, to replicate the success, profitability and near-monopoly of their Desktop Operating System and Office/"Productivity" tools. They <i>bought</i> the XBox market and supported big losses for many, many years. They don't dominate the games console market, they barely hold their own... Declaring "everything is a PC" doesn't fix the problem of the mis-match between Reality and their faulty perception.<br />
<br />
Their path <i>back</i> to near the top of the pack isn't doing "more of the same". They can follow Kodak and Xerox down the hill from market leader to irrelevance or worse with conservative "sound business strategies". These approaches have failed, the current senior management has failed, the current culture is dysfunctional and killing them not-so-slowly and the CEO and Board are letting it unfold.<br />
<br />
The only question for me is:<br />
<blockquote class="tr_bq">
<i>What would Bill do?</i></blockquote>
Bill Gates, whatever his faults, would've acted decisively long, long ago. We know he could turn the behemoth on a dime when needed: the <i>Internet Memo</i> where they abandoned MSN and embraced the 'Net, <i>everywhere</i>.<br />
<br />
This isn't a <i>technical</i> failure by a good business-person, this is a <i>business-failure</i> by an incompetent and egotistical CEO.<br />
<br />
Bill's not coming back, he's made that plain. By his actions, you'd think he foresaw this day more than a decade ago when he started converting his shares into cash... And Ballmer didn't.<br />
<br />
My thesis now is that Microsoft will squander its capital, both cash and "goodwill", on "more of the same" moves.<br />
<br />
If they follow pre-1990 IBM and walk further into denial by trying to maintain profits on falling market-share by inflating loyal customer charges, they will suffer the same fate as IBM: <i>a sudden and near complete collapse of their business.</i><br />
<br />
Will Bill be waiting there to pick up Office and Windows for a song?steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0tag:blogger.com,1999:blog-29583699.post-16954113268987177682013-06-02T16:35:00.001+10:002013-07-02T08:55:45.509+10:00NBN: Posts moved to a new blog. No more on this blog.This Blog was intended to be about "I.T." issues in general, but has become overtaken by the NBN.<br />
Posts and Comments <a href="http://stevej-on-nbn.blogspot.com.au/">have been moved to a dedicated Blog</a>...<br />
Please update any feeds you might have, assuming you want.<br />
<br />
I'll be removing the NBN posts from this Blog down the line.steve jenkinhttp://www.blogger.com/profile/16064724730975745470noreply@blogger.com0