Internet Changes Everything: Newspapers

The News Broadsheet was a pivotal element of the 1776 American Revolution, eventually becoming enshrined in the First Amendment to the US Constitution.

Newspapers were integral to the Twentieth Century rise and evolution of Western Democracies. Without "frank and fearless" reporting (and an engaged electorate), governments can quickly spiral out of control.

We're now 15+ years into the "Internet Revolution", so where are Newspapers in their journey on-line? Significantly, nobody seems to have discovered a "secret sauce" to generally monetise News and the work of Journalists in the way that Amazon and Google etc have monetised books and on-line ads. Those who make money from writing seem to do so from direct subscriptions - the on-line form of "newsletters".

In my first job, there was a direct and obvious connection between the 5g +/- 0.01g of sugar being analysed and 'the business': Our analyses determined payments for 500 or 1,000 tonne lots. Getting it wrong wasn't an option. My place in the scheme of things was self evident.

The world of Software and I.T., even after 60 years, still doesn't have that direct & obvious link.
I.T. shares many traits with Journalism:

• Both are "Performance Disciplines", like Music, Art, Surgery and Gymnastics.
• "Effortless Performances" (as in 'making it look easy') take a lot of skill and experience. The public and more often now, management, have little appreciation of the process & skills.
  
• Input Effort and Results bear no discernible relationship.
  
• Quality is Everything, but seems impossible to measure.
• Although both are central and necessary to the businesses they support, they are managed as "Cost Centres", with seemingly no attempts at connecting outputs with Profit.
• Both deal in intangible and invisible "stuff" - information. Often with tight deadlines and very fast decay in product "usefulness".
  
• Both share a central problem: Effort/Inputs are decoupled from Income/Results.

To research this, I chose "Fairfax Media" (ASX:FXJ), a major Australian player, because some of their "Mastheads" are over 100 years and "The Sydney Morning Herald" used to be legendary - known worldwide for its "Rivers of Gold".

Getting even rough numbers to judge demand and price-points seems very hard.
For any on-line business to succeed, revenues have to support costs. While serving bit-streams may be considerably cheaper than printing & shipping paper, what will people pay for it and how do you get money off them? How do you draw in more subscribers - what are your Marketing & Sales channels?

Newsagents keep the full "cover price" of most newspapers. For magazines and most other products, the publisher gets half with the distributor & newspaper splitting the rest.

The content of newspaper, News etc, is why people buy newspapers.
The price people are willing to pay for the whole paper indicates the "ultility" they get.
But how do you arrive at a value-in-use of just the News component?
Are there synergist effects in operation?

The output of Journalists, "News", is essentially a "Free Good" to both consumers and the business. To the publisher, each newspaper printed is only a cost - which they actively attempt to minimise, whilst revenues are from advertising and independent of pages printed. The connection is the "rate card" - people will pay more in advertising for wider circulation & readership. The connection is anything but direct and immediate. Meanwhile, "content" (& "classified" adverts) brings in readers, but the supply-demand curve is generally unknown.

Makes the economics of "free" community publications more obvious. The distribution costs are only slightly more than to newsagents and the claimed readership is maximised.

It also says why businesses are so very happy with "advertising only" publications, like "Trading Post". Pure profit and no content producers to wrangle! These business translate on-line very well - but lose all "display advertising" to corporates like retailers looking for mass market advertising.

Audited Circulation figures aren't too hard to come by and "The Press Council" publish a good snapshot of the whole Print Media scene, but there are only tangential references to journalist "head counts". Fairfax journalists feel pressured by staff cuts and are taking action.

To come up with viable business models for on-line News, especially when competing with "Free" on-line services like Google or Free-To-Air broadcasts (radio & TV), you need both sides of the Accounting Equation:

Profit = Revenues - Expenses.

Other marketing data is needed to determine whether Size Matters (only one global Google and Amazon) or Little & Local works or some combination in between...

Whatever the result, there is one guaranteed loser: newsagents.
They are the traditional Marketing & Sales Channel for newspaper.
What becomes of them in an on-line world?
Can their access to "passing trade" and existing business relationships be leveraged?

What we do know is that people are very happy with "Free" on-line content: using search engines to point to unbilled on-line newspaper articles.
Newspapers initially tried to force reader registration, even though content was "free" people went elsewhere. Fees were charged to access "archives" or additional material made available to paid subscribers.
News Ltd's upcoming experiment in charging for on-line content will be watched very carefully throughout the industry.

What would it cost to run an adequate newsroom? Do the 15 journos of brisbanetimes.com.au reported by the Press Council provide adequate local coverage? Various sources suggest major metropolitan news have 200-300 journalists, while the national broadcaster, the ABC, have 700 people in their "News Division" (which is how many journalists?).

Any 24/7 operation requires shift-work. To provide a minimum staffing of 2 people, needs a team of ~12. Numbers build very quickly as more sections & coverage is needed.

Fairfax's Annual Report (2007) says they made a profit of ~A$500M ($447M EBITDA) on A$2.3B sales.
Around one third of sales were from New Zealand. Overseas and non-print revenues were unclear.
The Australian Digital operation made A$37M on sales of $137M.

Their ~10,000 employees were their largest single expense: A$700M.
Paper and ink came in second at A$270M.
Sales and Promotions were A$88M.
Communications: A$17.5M
I.T.: A$15M and
News services: A$12M

Fairfax reports $8B of Assets - $6B of which is "intangibles" - "goodwill" and "value of mastheads". They've around A$850M in Property, Plant & Equipment - the physical assets needed to produce newspapers.

But how many journalists were there and what does a single "high-quality" newsroom cost to run? That's not going to change for an on-line News service.

What will an average subscriber pay for an on-line News service?
What additional content are needed, and what synergies exist?
How many on-line subscribers will sign-up for each different offering?
Can the existing subscribers be converted to on-line subscribers? What would help in the transition?
What are the different attributes that subscribers value and what premiums will they pay?

This says that monetising on-line News services may be hard and their general lack says solutions are still not obvious.

---

Other comments, observations and relevant factoids:

Newspapers serve important subsidiary functions, like being "the paper of record" for Births, Deaths and Marriages as well as public events, political speeches and disasters, crises and more.
"Public Notices" of many types are published - from bankrupts to probate on wills to personals.

Newspapers have come to be the definitive textual mass-communication device.
There appears to not yet be any on-line equivalent.
In the USA, recent failures of long-running mastheads says there isn't that much time left to find a good answer.

Circulations:
The SMH sells ~212,000 copies Mon-Fri (@ $1.50), 364,000 on Sat ($2) and [Sun-Herald] 505,000 on Sun. ($2?). It has increased circulation in recent years. No figures available for pages of advertising sold.
The Melbourne Age has comparable sales figures, though Sunday sales are roughly half the SMH.

'Readership' of the SMH is estimated at 853,000 Mon-Fri and 1,116,000 Sat.
Papers are shared around.

These figures suggest that newsagents & supermarkets make around $1.5M/week for Mon-Fri sales and the same again on weekends. Removing that income stream would leave a very unhappy sales channel...

Payments and Subscriptions types:

• Pre-paid or billed subscriptions. Like Home Delivery
• On-demand use: Ad-hoc or occasional purchase
  
• Prepaid access to articles.
  
• Business or family sharing. Limited copies shared between many.
• Public access - libraries.
  
• "Media Monitors" - clipping services across many sources relevant to a business.
  

What do Newspapers do?

• Inform
• Educate
• Entertain, and
• Surprise and Delight.

Newspapers tell you things you need to know, especially things you didn't realise you needed to know.

What comprises 'stories'?

• facts
• expert analysis
• commentary
• opinion, and
• interviews

Professional journalism brings brevity and precision. Stories are {complete, correct, concise} and hopefully {consistent}.

The value to the reader is collection and pre-screening: reducing a mountain of data and facts to quickly and easily accessible information. Journalists classify and prioritise stories, letting the reader minimise time used and maximise information found.

This "Usability" principle extends to presentation, layout and story structure.
The fonts and columns widths are chosen to best suit human factors.
Modern printing added pictures to text - an important aid to readers and writers alike.
The use of white space, headlines and graphics/pictures increase readability & accessibility.
Stories follow the "inverted pyramid" - the most important facts first, tailing off. It means readers can quickly scan articles and find the most relevant/useful to them, and sub-editors can easily trim articles to fit by removing trailing paragraphs.

It is instructive to read a 100-yo paper without these modern features - they look dense and impenetrable. The small number of pages would've been a blessing.

Professional Journalists bring special factors to collecting stories:

• access to people & organisations, like police, politicians, CEO's, ...
  
• funding for travel, communication and fees - like FOI requests or corporate data.
  
• Researchers, archives and access to expensive subscription services.

The users of News services look for many different benefits and uses:

• text, images, audio and video
• Mobile access
• Alerts and "Instantaneous news items" (the latest stories)
• Distraction, Relaxation and passing-time (as in commuting)
• Social settings - Cafe and Brunch
• Search and information: Browsing classified and focused searches

On-line challenges:

• distribution format. PDF's or HTML?
  How do I read it on the train?
  Does it read well on a laptop or iPhone?
  
• enforcing DRM. How to limit copying? What to do if copied illegally?
  
• on-going subscription rights:
  If I've bought a "paper" once, do I have permanent access to it, even after my subscription ends?
  
• Competing with Free-To-Net services. Seems hard, have to provide additional value.
  
• Catering for local community news
• Accepting input from the general public.
• The Blogosphere and non-professional writers. They can't be held to journalistic standards and ethics, can't be reprimanded or censured and will publish/repeat unsubstantiated gossip and rumour.
  

"non-News" Newspaper functions that Online services may need to duplicate:

• "paper of record" function: Public Notices, Births-Deaths-Marriages
• Mass-market display advertising.
• Marketing and Sales channels. How to grow new business?
• Access to Printed copies - eg. weekly summaries.
• Searchable service provider directories and classified adverts.
  Implies RSS-style alerts & monitoring.
  

On-line service challenges:

• Monetisation. Advertising, subscription or sponsor based? Other?
  
• Niche marketing. By location, interest, community, employment sector.
• Tiered Subscriptions: free, basic, premium, target area, search tools, ...
  Profits can be maximised by segmenting services with multiple price-points.
• Organisational access.

Telco pricing and market 'price elasticity'

There's a counter-intuitive effect with marginal cost of Production Factors, like energy (and Teleco services) - using the factor more efficiently, consumes more of the resource. Because you make more profit, lower prices, produce more and demand for the resource increases. The Khazzoom-Brookes Postulate/Jevons Paradox:

"energy efficiency improvements that, on the broadest considerations, are economically justified at the microlevel, lead to higher levels of energy consumption at the macrolevel."

The structural reason is simple: the market is highly price elastic, so decreasing prices a little lifts total sales considerably. In economics, this is a well solved problem for non-monopoly markets, "Profit Maximisation" occurs when MR = MR (Marginal Revenue equals Marginal Costs). [For monopolies, MR = 2*MC, IIRC.]

In the 70's & 80's at O.T.C., we made record profits each and every year - by exceptional marketing and sales strategy, which included dropping prices every year. [The TV adverts series, like the 'Memories' campaign, won many awards.]

This was driven by the technology: Moore's Law drove the per unit cost of services in both cable and satellite down exponentially.

Profits margins increased because full cost reductions weren't passed on. By the mid-80's it was cheaper from the East Coast to call London or New York than use Telstra to call Perth ($1/min).

This is a lesson Telstra Management never learnt and was obviously lost when O.T.C. was subsumed into 'the Borg' in 1992: passing on part of the Moore's Law savings, Revenue and Profits both increase.

Economic theory is very clear on this point:
Traditional (Premium) Telco Pricing isn't just 'bastardry', it kills your profits which will eventually kill your company.

People love to talk/communicate, it is one of the defining characteristics of Homo Sapien.
This underpins the profitability and constantly growing demand for Telco services... This, the consumer demand, is the real 'crown jewels' (or 'birth-right') of Telco's, not their networks and technology.

Nobody has yet plumbed the limits of the price elasticity curve for human communications.
Setting the marginal cost of phone calls to zero doesn't kill your profits as shown by the US 'free call' areas.
It does lead to changed behaviours - more calls, many shorter.

Conclusion:
The incumbent Australian phone companies, Telstra, Optus & friends, don't understand the fundamentals of their business:

• Moore's Law has been driving down their input costs exponentially for 4-5 decades, yet they haven't passed on those savings. The differential is now 100-10,000 (guesstimate).
  
• The demand for human communication is close to infinite.
• The Comms market has close to infinite price elasticity.
• Profit Maximisation is simple and 'Premium Pricing' (what-the-market-will-bear) is exactly wrong.

The challenge for the Telcos now, especially Telstra, is unwinding from their 'Premium Pricing' to a rational 'Profit Maximisation' model. How do they explain price reduction of 10-100 times?

The consumer backlash, if not managed, will be savage and punitative. The original 1990's duopoly Telco's, Telstra and Optus, will suffer greatly - to the point of being endangered.

"You can fool some of the people all of the time" - but woe betide you when those you've fooled understand they've been gouged/conned and have alternatives.

Aus High Speed Broadband: Barriers and Challenges

A response to the "Public Sphere" run by Senator Lundy on "High Bandwidth for Australia".
Previous comments on the National Broadband Network - about excess costs, poor design and addressing lost strategic opportunities.

Future posts will be linked from here.

Roger Clarke finished his presentation with a comment something like "the upside is obvious, the downside needs to talked about".

Without bringing down the tone, optimism and vision of the event, enumerating the Barriers and Challenges associated with roll-out and adoption of 'universal' High Speed Broadband (HSB) in Australia allows for generating better designs/options, avoiding 'surprises' and anticipating routes around and out-of "bear pits", "Forewarned is forearmed". Who knows, new insights may emerge from this viewpoint.

Alan Kay in his 2003 Turning Award speech identified the central importance of ICT:

"our field is a field that's the next great 500-year idea after the printing press"

Getting the transition into High-Speed right enough is important:
• There is no way back.

Setting user expectations is very important:
• Servers, not the network, will become the new bottleneck.

The central technical challenges are:
• Usability, Performance, Security, Reliability/Robustness.
Scaling-up and scaling-out current solutions won't be easy, obvious or simple. Probably not cheap or quick either.

The "drop-dead" challenges are non-technical:
• Social, Legal/Political, Regulatory, Cost.

User acceptance:
• Social, Legal/Political, Regulatory, Cost.

Australia has a history of slow take-up of new technologies. It took until 1975 for Colour TV to arrive in Australia. This is counter-intuitive because Australians are well known innovators, inventors and scientists. Individually we embrace change, collectively reject it.

Some of the central problems & differences with The Net are:

• universal & global access
• Uncertain Identities - "nobody knows you're a dog"
• non-physical, untraceable digital goods
• The Permanent Digital Record
  

Topics:

• Libraries
• Copyright, copying digital docs (c.f. photocopies & royalties).
  
• Content: DRM, Protection, Enforcement, "free for domestic use", "Free for Mash-Ups"?
  
• Creative Commons & shared content/resources
  
• e-Commerce and Monetising Content:
  
  • Royalties: Registration, Recognition, Collection, Distribution
    
  • Definition, Usage Rights, Nr Plays, Time-shifting, Time-to-live & View-Time
    
  • 'Sharing'
    
• Costs
  
  • $: up-front/install, on-going & operations, upgrade/variation, usage (Volume/Time/access), 'surprise' charges: excess-data, reverse-charging, malware induced
    
  • Damage and OH&S issues: e.g. RSI, eye-strain, ...
    
  • Availability: Regional, Speed, Filtering
  • User Time: Amplifying User Effectiveness, time cost of {Admin, Tools, machines}, Training, PC maintenance/modification/upgrade, Home Network {design, install, maint, upgrade}
    
  • Server Access: Free local on-net access, free home servers, Caching {Cost, access, copyright}
    
  • Security & Privacy: Admin, monitoring, repair
  • Time Shift Streaming:
    
• "Acceptable Use Policies", Netiquette, defining & enforcing "Rules of Conduct" {email, IM/twitter, blogs, web pages}
  
• Digital Divide: Socio-economic, Cultural, Geographic, Age, Disability
  • Digital Natives vs Gen X vs Baby Boomers vs Pre-boomers
  • Electoral Disenfranchisement through link/computer speed, proprietary file formats/browsers/utilities and codecs
    
• Smart Networks vs "Too smart by half" - limiting and inflexible implementations/options
  
• Power (Electricity), Mobility, Screen Sizes (iPhone, Kindle, netbook, laptop, desktop, Video)
  
• "All About the Data": Guaranteed Open Standards and Open Access to Public Content. {Audit, enforcement, appeal}
  
• Unintended Consequences: Recognition, Identification, Redress, Enforce/Change, Legal suits
• Privacy and malicious content: Erasing 'leaks' and the untrue/misleading from The Permanent Digital Record.
• High Performance networking: Video, Adult Content and Gambling.
• SPAM, phising, malware, etc:
  • Cyberspace is a unique and different Legal Jurisdiction. Global approaches, like "The Law of the Sea", Intellectual Property and 'Post and Telegraph' are needed. A critical enabler is physical identification and tracking of maldoers and universal extradition: "nowhere to hide" has to be both a mantra and a reality.
    Bringing recalcitrant regimes into line shouldn't be hard: disconnect or rate-limit Internet to their country.
    
  • Digital Organised Crime and Digital Conspiracy are serious, global offences which are in dire need of addressing. Underpinning legal and properly funded policing frameworks are needed. Unlike the US "War on Drugs" and "on Terror", th
    
  • Identity Theft is a major crime & needs one international policing body, with an associated court and realistic penalties.
  • Falsified E-mail Identity, the basis of SPAM, is as illegal and serious as Mail Fraud. It needs to be treated so, with efficient detection and rapid response.
    
  • Theft of resources: bandwidth, CPU or storage, is as much theft as stealing Fax paper. There was a landmark case in the UK where adverts using unsolicited Faxes was successfully prosecuted. Internationally, the same precedent is urgently needed to control 'botnets' and other malware.
    
  • Stealing bank login details and so money is exactly Digital Bank Robbery. This is crime, pure and simple, already recognised by every jurisdiction. Why there seems to be no Political will to identify and rapidly bring-to-book these criminals I find astounding and indefensible. The mischief and damage possible through Digital Robbery is enormous - if ever fully realised, 'shock and awe' will be appropriate.
    
• Government
  • The Net is a Distance and Time shifter: it changes the nature and style of public interactions with their elected representatives. Importantly, it changes representative accountability.
    
  • A single/integrated approach for all levels of Government (Local, State, Federal) is required for best results.
  • Public access to appropriate Bureaucrats requires both a role/positional Directory and 'ticket handling' systems like Helpdesks. The public have a reasonable expectation to prompt, efficient resolution of requests, complaint etc with Open and Transparent reporting of performances.
    
  • On-line anonymous venues are the perfect mechanism for whistle-blowers and other public-interest disclosures. Handling problems like the Rockhampton "Dr. Death" allegations or the "Butcher of Bega" sexual attacks, while avoiding false or malicious rumour is the key test.
    
• Addressing Human Limits on Input & Output
  • John Mashey, creator of the MIPS chip, observed the disparate bit-rates for Human Input & Output. Typing is incredibly slow (30-40bps) speaking & listening relatively fast (10-32kbps), but we excel at inputting vision (10-100Mbps).
  • 
    
• more...
  
  
  

The issue of seeding and supporting early adopters and leveraging their experience/skills into the general populace is central to the wide-spread adoption and use of the new technologies. "Who you gonna call?" when you need to know how to do setup/do something is a critical support mechanism.

Alan Kay has some insight in this problem as applied to Education, with additional comments on applying to adoption of new Computing paradigms. A feasible approach would seem small, focused communities of interest who support & reinforce one-another and who 'spread the word'. Something like the CRC program for Research.

Why are we doing this?

"Augmentation", an important concept, was mentioned by a later speaker. (James Delow?)

Doug Englebart, one of the early giants in Computing, ran SRI's Augmentation Research Centre.
In 1968 he and his group gave an extraordinary demonstration for the ACM - it included hypertext, interactive video and teleconferencing. Implementing this same demo forty years on would still be a challenge.

In 1945 Vannevar Bush, the inventor of Hypertext, designed a system, "The Memex", to augment/amplify a researchers ability.
Key to its use and value was the ability to share 'Associative trails'. It allowed "researchers" to identify and exchange exact lines of research - all the material in one place.

Do our current regimes of copyright and DRM support or hinder the widespread use of technologies like this?

Homo sapien has a need for freedom and meaning.
What marks us as unique is our thirst for knowledge, the need to 'understand' and to communicate. "Solitary Confinement" is an especial punishment.

We cherish stories and story-telling. Maslow even identified "self actualisation" as our highest need. We don't just learn, we record History and identify its Lessons - what works, what doesn't. We advance because we "stand on the shoulders of Giants".

High Speed Broadband is about Humans limitless need to Learn and Communicate.
It is the lastest, but not the last, technological means to facilitate our many needs.
Not the least, our daily work, trade and commerce.

Which segues to our individual and collective 'darkside'.
Creating a new tool/mechanism means creating effective non-draconian ways to keep it safe.
Taking the stance this brave new frontier should be Free and Unfettered is amazingly naive and self-serving. If it were this simple, there'd be no SPAM or malware. Much worse is waiting out there.

Alongside the technical and organisational challenges, we have to consider the Regulatory and Legal environment of what is acceptable behaviour and 'words'. Then figure out how to police and enforce those limits, without destroying the very thing we are trying to foster.

Universal access high-speed broadband isn't "just faster", it's a game changer in exactly the same way that motor vehicles redefined our societies. Megatropolis's are not possible without high efficient, high performance transport networks. Our advanced technologies aren't possible without those large conurbations, either.

"More of the same" thinking isn't adequate for this new technology.

A useful beginning is to ask the questions:

• What do we need to do MORE of and
  
• LESS of?
• What do we need to STOP doing,
• START doing, or
• CHANGE/MODIFY what we are doing?
  

It's a Brave New World. We can try to get it off to a good start.

Broadband Lost Opportunities: Gas and Broadband

1999/2000 saw the laying of the "Eastern Gas Pipeline" (EGP) - 795km from Longford in Victoria to Horsley Park in Sydney, at around $450M. It carries natural gas from Gippsland Basin (and Bass Strait?) and provides additional supply security to NSW.

The mainline route is:

close to the towns of Bairnsdale, Orbost and the Cann River, before turning north past Bombala and Cooma. From the eastside of the Snowy Mountain region, it travels northeast to the coast through Nowra, Port Kembla, Wilton and on to Horsley Park to its termination point on the outskirts of Sydney.

Country Energy Gas is the NSW regional retailer. The Energy Networks Australia State Summary say this about the NSW country towns connected to Natural Gas:

There are 24 210 km of reticulation mains serving 901 000 customers in the State’s major urban areas, and its large and small regional centres.

Wagga Wagga, Tumut, Adelong, Cooma, Bombala and Gundagai, Albury
Nowra, Bomaderry and Queanbeyan.

Some of NSW's gas pipeline "Laterals" are listed in this NCC PDF file:

• the Mainline from Moomba to Wilton;
• the Dalton to Canberra lateral;
• the lateral pipeline from Young to Cootamundra and Wagga Wagga;
• lateral pipelines from Young to Bathurst, Orange, Lithgow and Oberon; and
• the lateral pipeline from Burnt Creek to Junee, Griffith, Leeton and Narrandera;

The Sydney Hydraulic Power Company (now part of KONE Elelvators) pipe network was shutdown in 1975 after nearly a century of operation. The buried high-pressure water pipe through the CBD remained a valuable asset. In the last 1990's it was purportedly acquired for Telecommunications use (no references found).

Utility companies have been mindful, since the deregulation of Telecommunications in Australia, of their unique access to households/businesses and the commercial possibilities flowing from it. PowerTel, formed in 1998, was 31% owned by the "Downtown Utilities" consortium, EnergyAustralia, Citipower and Energex. They provided the local loop right-of-way to a potential 2.5M of their customers.

Transact, in 1999/2000, also leveraged this access from powerlines to roll-out its network.

Rail corridors have also been used (extensively?) to lay optical fibre cheaply. A very brief sample:

• In 2001, Leighton Holdings announced the 1800km Queensland "Reef Network" costing A$85M installed in the QR corridor. ($45/m)
  
• QR Networks is currently selling spare capacity, dark fibre and site/corridor access.
• VicTrack also has a Telecommunications arm.

Reef Networks' $45/m cost is well under the NBN's estimated ducting only costs of $60-$150/m.

The various Gas Networks should've been aware of the these options and had the opportunity to leverage their extensive pipe network upgrades and additions into both long-distance and local-loop Telecommunications networks. Only in one town, Cooma, did a third party partner with them to lay empty conduit/pipe when the reticulation network was constructed. The marginal cost of laying another pipe in the same trench was minor.

This oversight has massive implications now with the (A$43Bn) NBN roll-out.
Not only would those utilities have an additional on-going revenue renting duct access, it would've made the NBN roll-out much cheaper, easier and faster. Nobody wants their street dug up one more time.

The shareholders can't be happy with their boards either. The real loss was in National strategic and commercial competitiveness. A lack of vision and execution by the responsible regulators has cost the country dearly.

Phone Companies vs Broadband

Traditional Telcos, "phone companies", are exactly the wrong people to own and operate consumer Broadband networks, and their close relative, Cable TV.

There is an inherent conflict:

• Telco Engineering is very different to Computer Data Network Engineering. You cannot be good at both or build networks that are good at both.
  The network & equipment designs and tradeoffs are antithetical and antagonistic.
  
• Telcos make money from phone services, which are relatively low bit-rate. It is against their commercial interests to sell high-quality, high-speed data networks at affordable prices.
  To expect them to cannibalise their most profitable line of business is asking for incompetent & irresponsible management, which puts them in jeopardy from corporate regulators and malpractice suits from shareholders.
  

There has always been a fundamental conflict of interest in any Telco rolling out Internet broadband services: cost-per-bit charging either makes telephony effectively free or streaming TV monumentally expensive.

In Australia after Telco deregulation, the initial duopoly (Telstra & Optus) were allowed to roll-out both Broadband and Cable TV. The early 1990's Cable TV debacle with 80+% duplication showed both the inherent commercial conflict and that both Telcos were aware of it and would act, aggressively, to maintain their commercial interest.

The current parlous state of play with National broadband service underlines the argument.
The design of the new NBN by Telcos is similarly flawed. They need to make it expensive and complex to protect their phone networks for as long as possible.

That our legislators and regulators have failed to understand this inherent conflict, to see the massive market distortions/failures is an amazing oversight. There are either blind to it or accepting of it and in the thrall of the large Telcos. Is it ignorance, incompetence, arrogance, apathy or something worse?

The solution is clear and demonstrably effective:

• Issue separate licences for telephony and data networking, with the same cross-ownership restrictions/regulations already applied to the Media.
  Papers, radio and TV need to compete with one another.
  Regulations prohibit a single marketplace being dominated by a sole player.

Explaining why Australian consumers aren't served by a protective communications regime is becoming increasingly hard...

Telco Engineering vs Network Engineering

For many years I had the uneasy feeling that the Telco Engineers I worked with/for over most of a decade at O.T.C. did not actually understand Computer Data Networks.

Here I attempt to formalise and explain that thought.
The implications/ramifications won't be examined in this piece.

History

Telephone/Telco Engineering dates back to between ~1880 with the formation of the first phone companies and 1915 with the first US transcontinental (long-distance) calls. Dial phones were introduced in the US circa 1919, but automatic exchanges had been invented well before. The profession of Telco Enginering has around 125 years of tradition and practice.

As a profession, they have been very good at doing what they do - creating reliable point-to-point voice communication. They have extended into high-availability point-to-point digital services.

Modern Computer Data Network Engineering dates from around 1982 with cheap mini-computers and the IEEE 802.3 standard for Ethernet. Cheap PC's, affordable Interface cards and UTP (10Base-T), standardised in 1990, set the stage for current LAN's - desktop and server. This gives the modern discipline around a 25-year history.

The Universal Network Glue, IP (Internet Protocol(s)), dates from 1969 with the interconnection of the first two systems. Early Telco Data Networks go back to Telegraph (morse code) and Telex and ended with X.25.

For completeness, the World Wide Web, created by Tim Berners-Lee and Robert Cailliau at CERN, and now popularly referred to as 'The Internet', leveraged in 1989 PC's, ethernet and TCP/IP.

1996, the birth of the Modern widespread Internet, was marked by Microsoft abandoning it's proprietory MSN network & protocols and adopting Internet Everywhere.

Differences in Networks and Approach.

Telco networks started with patent wars, bleeding-edge technology and on-going & increasing requirements for large capital investments. That 'sunk cost' became huge as phone access was rolled out almost universally, at least in the 'First' and 'Second' world, providing a considerable barrier-to-entry for new players. After some decades, incumbents could easily kill new competitors by under-pricing them - they had paid for their networks and with great Free Cash Flow, could upgrade & extend their cable plant out solely from operating revenues.

To compete in the Telco world required huge investments in cable plant and switching equipment, and extensive, preferably full, network coverage. "Metcalfe's Law" states the value of a network increases with the square of the number of connections. A provider with a slightly better coverage in an area, all else being equal, quickly gained an economic advantage. Achieving better cash flow & profits either through higher charges or more subscribers. This could pay for faster expansion of the network, increasing their advantage. A virtuous circle.

Subscribers could not, even if they wanted, install & run their own cable plant & subscriber equipment. Initially, it was too expensive and patent-protected, then precluded by technical compliance requirements, then by legislation and regulation.

Telco operating principles became:

• effective monopolies per region
• hub-and-spoke design
• extensive overbuild to cater for projected demand
  
• high-availability, high-cost central equipment and interconnects
  
• simple subscriber equipment and complex exchanges and transmission systems
  
• 'Premium Pricing' model ("what the market will bear", vs "cost plus")
  

Telco networks are a classic Cost Accounting study: almost all costs are Fixed and Indirect, often dominated by Financing costs. There are almost no Direct or Variable costs.

E.g. the only marginal cost for any phone call is the cost of electricity: ~1 watt per phone. Around 20 milli-cents per hour. Capturing & processing billing data is 100-1000 times more expensive.

Telephony and Data Networks differ in almost all details, "Let me count the ways":

Factor	Telco	Data
connection	Circuit	Packets
speed	fixed	variable
Noise	ignored	error-correction
echoes	cancellation	n/a
multiplexing	external systems	inherent
Model	Central Switching	Distributed switching
Topology	Hub-and-Spoke	Buses, self-healing loops distributed equipment
High-Availability	component redundancy	whole switch duplication
congestion	no circuit, call fails	slower transfers
lost data	noise & dropouts	retransmit
switch design	non-blocking, continuous connection	queues, dropped packets, retransmit
lost data	noise & dropouts	retransmit
variable delay, jitter	highly sensitive	tolerant, retransmit
multiple connections	more links	increase link speed
Encryption	External, expensive	Embedded, Extensible
Intelligiblity and 'Quality of Service'	good, guaranteed	variable, no general QoS
Local loop Scalability	Rebuild, reinstall	In-place link upgrade
Upgrade	Long-range forecasting, Initial overbuild	In-place upgrade, incorp Technology advances
Equipment source	Specialised, expensive	generic, commodity
Financing	Large CapEx hidden in monthly rental	Prepaid install and Customer owned
Billing	Post-paid, unlimited Credit, Itemised bills	Prepaid with limits
Capacity	megabit range	HD video capable, tens of gigabits
Multicast	Only single-cast	multicast capable

What must be made clear:
there are some services that IP Data Networks do not currently deliver as well as the Telco networks. Those requiring low-latency, low-jitter, and low-noise. I.e. a guaranteed (high) Quality of Service. Even the traditional consumers of these services, radio and TV, are changed their work practices and moving to in-house IP networks or general Internet delivery.

Until 1999/2000 overseas telephony dominated those trunks. Since then, direct internet traffic has kept growing (exponentially - what doubling period?) and now swamps all other service demands.

As an example of the capacity differences: the ~10M landline services (at 32kbps) and 21M mobile services (at 9.6kpbs) represent a maximum of ~500Gbps demand, possible in a single, albeit large, router. The usual demand is around 2-5% of the maximum (10Gbps) - now well within the capability of low-cost routers.

Telco Engineering applied to Public Data Networks.

Large modern corporate Data Networks are "Pure Internet" networks. Networks such as the Department of Defence, cover most parts of Australia and extends overseas. It provides for 100,000+ desktops, a larger telephone networks, audio & video broadcast and secure services. They compete in size, service range and complexity with normal "common carrier" (Telco) networks.

If 'traditional' Telco Engineering approaches were more cost-effective or provided better availability & reliability, then they would be in use. The usual arguments for not delivering "Pure Internet" (commodity links/equipment, symmetrical upload/download) to households is population density. Defence faces the same distribution problems across its many, large campuses - and still run "Pure Internet". When cabling/trenching costs dominate, it still makes sense to run small copper or optical fibre cables with switching systems distributed through the network.

The costs of running individual copper or optical fibre from a central exchange to households increase dramatically over a commodity Ethernet/Internet solution:

• total physical copper or fibre required is 10-100 fold more.
• large cables (eg 200-pair) are expensive to buy, install, join and repair/maintain
• many joints, each a failure point, are required to each customer premises, versus a single clear run to a local access point
  
• duct sizes near the centre get very large, compounding the costs, complexity and maintenance/upgrade problems
  
• with copper, cross-talk & interference problems compound with increasing circuits
  
• generic, commodity equipment cannot be used in either the Central Office or subscriber
  
• link speeds are fixed unless a major equipment upgrade is performed.
  

The Transact and the HFC Cable TV networks utilised this 'network embedded switching equipment' approach, resulting in per-house-passed costs of under $2,000 vs the $5,500 of the proposed NBN.

Perhaps the most convincing argument is what the Telcos now use for their backbone networks: Pure Internet. Many of their new services offerings are managed services derived from this internal IP network.

Or ask what companies are best positioned to offer "Triple Play" (TV, Data/Internet, Phone).
People with high-bandwidth backbones and upgradeable local-loops.

NBN - Powerplay or 'for real'?

During the week I had a fault on my phone line and got to talk to the Telstra tech afterwards.

His view was naturally Telstra-centric, but contained wisdom & insight.
It would make sense to have just one local access loop with just one maintenance organisation to which all Telco's have equal access. And it couldn't be owned by any one or two commercial players, that distorts the market.

He described an asymmetry: how new Telco entrants can build their own infrastructure and deny access to all others, but Telstra was obliged to provide access to 'their' copper network to everyone. He sited the sad case of a pensioner needing a phone and waiting for quite sometime (and paying a big fee) while Telstra dug in a cable that the developer should've installed... All the time there was physical cable to the premises owned by another Telco, but inaccessible.

The 1994/5 HFC cable TV rollout by Optus & Telstra (80-90% duplication) shows the insanity of Telecomms commercial arrangements and regulation in Australia. Behaviour that you wouldn't tolerate in school children.

That it was never a commercial decision is shown by the subsequent huge write-offs by both players. If they'd be ordered to "play nice" and construct a single infrastructure with bilateral access, the face of Australian Telecomms would be fundamentally different today. Cable TV would be a real force - possibly covering 80% of houses and making real profits.

The Telco regulators have allowed the same pattern to be repeated with mobile phone operators. In the USA & Europe, operators allow competitors access to their networks and make good income from it. It's called "roaming" and is supported by all the standards, hardware and handsets...

Everyone benefits, it's a positive-sum game. It doesn't stop operators extending their networks when they know they'll make more money by building their own infrastructure. [They have hard data on their customer call patterns.] It also means new entrants are 'born global' and have time to build-out their network and manage their cash-flow and CapEx. It encourages and enables real competition, which again benefits everyone.

Only it doesn't happen in Australia. There are 4 or more independent networks, everywhere, and no roaming agreements. It's not about commerce or service, but sheer bloody mindedness. All these Telco's have roaming arrangements with overseas operators. They have both the technical and commercial knowledge to do local roaming. [There is some for mobile wireless internet.] Everybody loses - it's a negative-sum game.

Regulators should not allow "Coverage" to be a marketing differentiator.
Failure to cross-connect/access should lead to heavy fines and eventual revocation of a Telco license. Telso Licenses, like Banking, are granted so commercial entities to primarily provide a public service. In return, a limited monopoly is granted.

It's all about providing public services, not about the profits of licensees.

Back to the NBN.

What's the long-term impact on Telstra if Kevin Rudd & Co forcefully construct "The One True Local Access Network"? If I were Rudd and was forced down this path, I'd deny Telstra access for forcing the issue in the first place.

Everybody loses, comms prices are high, services are limited and national economic competitiveness declines.

There is a 'critical point' of market share at which Telstra cannot continue to maintain and operate a parallel, full-coverage network - even with more limited services. As their market share declines, their profits reduce, putting pressure on maintenance, operations and customer service. This leads to more limited offerings, poorer service and unhappy customers, leading to yet smaller market share, decreasing profits and an inevitable "death spiral" that can only be broken by massive capital injection or embracing the NBN.

Telstra embracing the enemy seems very unlikely. Over several decades, the Telsra Board and Management have demonstrated they will not work with others, sometimes even after ACCC action and court directions. The have proven to be obstinate and recalcitrant.

There is also the competitive services problem: if the NBN provides desirable services that Telstra cannot deliver, then an increasing number of people leave Telstra, leading to the 'death spiral' by another route. Telstra can buy market share by dropping prices, but at the price of longevity - they destroy their profits and the ability to fund growth & new services.

The Federal Government and its regulators must know these things.

Are they taking Telstra on head-on with the intention of putting it out of business? (It can only be a 'take no prisoners' struggle to be won by the deepest pockets or a change of political direction.) The last thing Australia needs is the NBN being privatised before it is the dominant player. Without unlimited financial backing, Telstra would win, even if mortally wounded itself.

Or is it just a Power Play to force the Telstra Board and Management to wise up and 'play nice'?

We'll only know in hindsight.
Australia shouldn't have to bear either the massive cost of duplicating the local access loop or of Telstra failing.

---

Update 1. Sunday May 3, 2009. Senator Kate Lundy points to this piece by Richard Alston (Minister for Communications etc 1996-2003). Alston notes that Telstra aggressively duplicated the Optus Cable TV roll-out. He doesn't say that as the responsible Minister that he could've acted to prevent or change that.

---

Update 2. Tuesday May 12, 2009. The Australian reports :

The federal Government will offer Telstra the chance to buy up to 49 per cent of its national broadband network, if it agrees to voluntarily hive off its wholesale arm.

Telstra undertakes "structural separation" - into Wholesale & Retail arms. In return for its current Fibre Network, gets 20% of NBC (National Broadband c/o).

With both a new CEO and a new Chair (now Donald McGauchie is gone), Telstra may be able to resile from its "never, ever separate" position.

That values Telstra's 'Fibre Network' at $8.6Bn and allows them another $12.5Bn investment.
After the original $4.7Bn for NBN 1.0, the Govt. needs an additional $12.4Bn - half public, half private.

Guesstimates for NBN components:

• Domestic subs, 9M @ 100Mbps ($1,500/house): $13.5Bn
  
• Exchanges, 1,000 @ ($500/sub + routers/uplink): $5-8Bn
  
• Backhaul/Interstate upgrades (10+Tbps scale): $5Bn
  
• International cables - 10Tbps (1Mbps/household): $5-$10Bn
• Peering & ISP interconnects (20-100): $2Bn
  
• Rural/Remote radio/Satellite, 1M @ 12Mbps: $5-10Bn
  
• Content provider feed network: $2Bn
• High bandwidth subs: Business, Schools, Hospitals, Govt: $5Bn
• Network Operations, Maintenance & Test spares/depots/equip, Training: $5Bn
• Billing, Call record & traffic analysis, Line management & other IT systems: $2Bn
  

---

Update 3. Friday May 15, 2009. Stephen Bartholomeusz in "Business Spectator" reports:

Lindsay Tanner has confirmed what was suspected. In arriving at its estimate that the cost of the revised national fibre-to-the-premises broadband network would cost $43 billion, the Rudd government essentially dreamed up a big number and then added to it. ....

The evaluation that should have preceded the commitment will now occur, with the government commissioning an "implementation study", which one assumes will consider the complex economic issues involved and come to a conclusion whether the NBN is a viable commercial proposition.

If Telstra agrees to "structural separation" & buys in, the economics change.
If not, there's enough money in the bucket to do this thing "right".

Mircosoft can't write O/S code

This cartoon lampoons Windows 7.0 Beta. Eerie.

It underlines for me that Microsoft is crap at writing Operating Systems code.
O/S's need to be correct, secure, robust (resilient to errors internal & hardware) first and foremost. Only after that look to features and 'performance'.

How come the Great Unwashed, the home & business users and the media seem blinded in the same way the fanboi's are?

Wikipedia's "Time Line of Operating Systems" underscores this point - compare Microsoft's offerings since 1999: Win-2000, Win-XP, Longhorn: abandoned, Vista and 'soon', Win-7.

Compare to Annual or Biennial shipping of new releases OS/X, AIX, Solaris, HP/UX, Ubuntu, Red-Hat & Fedora, SuSE, ...

Microsoft is the exception, not the rule. Everyone else in their class does better.

Apologists for MSFT may add Win-98 SE, Win-ME, Win-server 2003/8, Media Centre, Home Server, Small Business Server and even Win-CE - plus the many 'Service Packs'.
This is all confabulation. The code differs mainly in licensing restrictions and add-ons.

The 10,000 strong Microsoft 'team' put around 25,000 man-years into Longhorn before Jim Allchin decided to call it quits and start again with what became Vista...

Jim Allchin, who had overall responsibility for the development and delivery of Windows, explained how development of Longhorn was "crashing into the ground" due in large part to the haphazard methods by which features were introduced and integrated into the core of the operating system, without a clear focus on an end-product.

Producing commercial grade O/S's require discipline, design and defined processes - Software Engineering 101 - to pump out regular updates & upgrades. The professional challenge isn't in producing a good piece of code just once, but like the Space Shuttle - doing it again and again to Schedule, Cost, and near flawlessly.

That's also basic Project Management: Plan, Schedule, Control.
Decide what you're going to do, plan how & when you're going to execute the project, manage deviations/problems/surprises during execution.

Operating Systems are vitally important - they sit under every other software layer.
They set the upper-bound for Security, Reliability, Usability and Performance of all applications on the platform. Nothing can be more robust than the layers it sits atop.

Hardware Engineers might take issue. It's long been understood how to fabricate ultra-reliable hardware systems from redundant components. Nobody has paralleled that work in the O/S and Applications arena, so my assertions stand.

Don't get my criticism wrong. Microsoft and Windows have been important in creating the world of commodity computing as we know it. I'd even rate them good at producing some Applications, such as Office. (But the software does suffer from 'featurism' , another challenge.)

The first Microsoft O/S's were adequate and, supported with unrivalled marketing, brought in the age of ubiquitous, mass market computing.
Microsoft have leveraged their natural monopoly (read 'proprietory lock-in') better than any company I know of, in any field.

Their Marketing is an example to be emulated, unfortunately their production of O/S software isn't.

High-Speed Broadband: Excess Costs and Opportunity Losses

The National Broadband Network, NBN, is a chance to correct regulatory deficiencies, address broadband network design problems & high costs and ameliorate past strategic mis-steps.

• Gas network providers not leveraging their investment in 'holes in the ground' and high consumer connectivity was a huge strategic blunder.
• Allowing the 80% duplication of cable TV network was a massive failure of Government oversight. It would've been obvious within 2-3 months and should've resulted in quick, decisive action. As was not forcing 'roaming' between mobile service providers.
• The NBN design reflects that Telco Engineers don't do Data Networks well... Exactly the same way you don't build a water, electricity or gas network as if its a phone network (hub-and-spoke).
• The technology choices & network design seem wrong-headed to me. They are not using off-the-shelf commodity hardware, but very expensive 'Carrier grade' offerings with limited scope for upgrade & expansion.

What seems missing in the designs I've seen is redundancy and self-healing...

Optical Fibre is very good for running in 'loops', and the simplest arrangement being 'counter rotating rings'. If one segment is broken, traffic can still reach the other side by going the other way around the ring/loop.

The NBN install will be the main broadband infrastructure and "local-loop" for the next 30-50 years. It's an opportunity that can't be let slip. And one that the public deserves good value for its $43Bn.

Costs of NBN - National Broadband Network, Fibre-to-the-Home

Connecting Australia with broadband/fibre systems is closer to implementing Cable TV than a new Phone network. The $43B estimate is 3-5 times too high based on Cable TV rollouts.

That difference completely changes the economics of the project.

The $43B NBN estimate is for 90% houses connected, or ~$5,500 per house passed.

Stewart Fist estimated in an article, 'Pipe Dreams', in 1996, that it would cost $10Bn to roll-out cable TV to 66% of houses. That's not the 90% of the NBN, but indicates a factor of three difference in price.

Queensland Government in Project Vista ($550M) was looking at around $3,000/house, as was ConnectSEQ ($300M for '3 million people' - for perhaps 1M houses)

The Victorian Governments "Project Aurora" was estimating $1500/lot in a new subdivision, confirming the QLD ~$3000/house.

The Bis Shrapnel report to the ACCC in 2001 on "Telecommunications Infrastructure in Australia" cites:

• TransActs' costs for fibre (phone & TV) and DSL services to 100,000 homes in Canberra as $80M - under $1,000 per house passed.
• Optus HFC cable rollout (95-97) at $3,000M for 2.25M houses passed, and
  
• Telstra's cable (94-97) $4,000M for 2.5M houses passed (underground cabling)

Circa 1995, costs were under $2,000/house for either aerial (Optus) or underground (Telstra).

In the Telstra/Optus roll-out, there's an 80% duplication factor. Optus, who started second, used a duplication factor of 10-15% in their Business Case. The inference I draw is they did not cause the massive, irrational and uneconomic duplication.

Elsewhere, a government report notes that years later Telstra wrote-down its cable network by ~$960M and Optus by $1.4B. There was also considerable litigation by Optus is gaining access to Foxtel content on its network. (Telstra had originally gained an exclusive distribution agreement.)

Moore's Law, mass manufacture and advances in lasers & optical fibre/jointing/laying since 1995 should've brought down the equipment & install costs 2-5 times in real terms.
Widespread use & availability of horizontal boring should've decreased underground laying costs by a large margin. In 1994, it was considered a novel technique in Australia.

Neighbourhood Cable offers cable TV and 50Mbps in Mildura, Ballarat and Geelong. One article claims it cost $65M for 32,000 homes in Ballarat: $2,000/house.

Wikipedia lists a number of existing fibre-to-the-home deployments in Australia.

Summary:
The economics have to work if there are already companies providing FTTH services.

Historical costs of large scale cable rollouts are seriously at odds (by factor of 3-4) with the estimated $43Bn.

Something is wrong, but nobody is saying why the NBN estimates are so high.
What is being bought for the additional $25-$30B apparent overspend??

---

Canberra Region initiatives.

Around 2000, Adrian Blake ran an ISP in Cooma (2,600 houses, 300 businesses). He paid AGL to include 80km of plastic pipe in the trenches when gas was laid. (Fibre was to be later pulled in.) Promised funding failed to materialise when the 'dot boom' turned into the 'dot bust' and the ISP went into liquidation. The empty pipe asset was finally purchased by Telstra but not used to deliver domestic services.

TransACT rolled out a Fibre-to-the-node cable TV, broadband and telephone network over the aerial Electricity poles owned by its parent company, ACTEW (later merged with AGL). The key point of difference was 50Mbps VDSL over the short copper tails to support streaming TV and broadband.
In 1998, TransACT started a trial in the suburb of Aranda and was to fully deploy its network around 2000. It too was caught up by funding problems related to the 'dot bust'. It seems the rollout stopped in 2003.
Robin Eckermann was the original TransACT Chief Architect

In 1995, Telstra announced it would spend $20-30M laying 'fibre-to-the-kerb' in the new town of Gungahlin. Anthony Goonan, General Manager for Strategic Marketing for Telstra in Canberra, was interviewed by the ABC about the project at the time. The initiative seems to have sunk without trace after the announcement...

The ACT Government in 2004 ran a major project upgrading its internal network to 200 sites around the Territory over a private fibre network. Later upgrades included moving its 12,000 telephones from Telstra to VoIP running on this backbone.

ICON (Intra Government Communications Network) provides point-to-point dark fibre between Australian Government agencies around Canberra. ICON started in the mid-90's by DFAT (Foreign Affairs and Trade) and Defence for links between their systems. Management has moved from DFAT to Dept. Finance and Administration.

Death by Success

Being too successful leads to failure unless you are aware of the problem and carefully monitor and protect against it. Every large company faces this problem - in I.T. for example, Google and Microsoft.

The rule applies at all levels - individuals, projects/teams, companies and countries. Even potentially our species and the whole living planet.

I was reminded again of this yesterday by Victor Cook in "The New Battle for Your Desktop" where he analyses the financials of Google and Microsoft (GOOG & MSFT). He describes a problem with Microsoft:

The company actually captured 77.5% of combined revenues in the quarter ended June 30, 2007.

Sometimes it's great to be the market leader. But in this case it's not. Microsoft shelled out $539 million for that 77th share point. And it was worth only $172 million after total costs. Not a good thing. To maximize earnings (by optimizing total costs) the company should have captured only 60.3% of the market. That's the point where the marginal cost and earnings per share point were exactly equal.

Microsoft's earnings were $3.991 billion at its actual revenue share of 77.5%. At the optimum level of competitive spending Microsoft's revenue share should be 60.3%, generating earnings of $6.271 billion. The difference is the $2.3 billion in theoretical earnings that management threw down the drain in the 2nd quarter of 2007.

Cook analyses Google in the next piece in the series and concludes they'd been under-spending into 2007, but were correcting it. (Piece is "Blue vs. Red Ocean Earnings Productivity")

Economics 101 - or What's behind this apparent contradiction?

If Microsoft had 100% of the market, a pure monopoly, wouldn't that maximise income and hence profits?

There are three issues to address in there:

• income isn't profit. (Profit = Income - Expenses)
  Max income may cause a loss if the expenses in earning it are too high.
  
• Maximum profit occurs when the cost of the last dollar of income earned equals the cost of gaining that dollar (as in sales/marketing/production&distribution).
  MR = MC. (Marginal Revenue = Marginal Cost)
  
• Monopolies have a different problem, as there are no 'substitutes' for their products and not other players in their market.
  Do they charge a single-price or not?
  Effectively this creates a series of markets which compete at the change-over price.
  

People won't pay 'anything' for a good that's not necessary to life: there is a Price/Demand curve. A small number of people will pay a high price, and everyone will have it 'for free'. The trick is knowing the shape of the Demand Curve (because that's external) and matching that to your Supply/Cost curve (you should know this, its internal).

Wikipedia has good introductory pieces on these topics:
Marginal Revenue, Marginal Cost, Profit Maximisation, Price Elasticity of Demand and Mirco-Economics. A piece on Monopolies discusses the "The single price monopoly profit maximization problem" - which states that price should be twice the production cost (MR = 2 * MC).
It also introduces the notions of "Consumer Surplus", "Producer Surplus" and "Deadweight Loss".

Note that while the Profit Maximisation rule is invariant - it applies to all operating companies - it is just one area needing close attention/management in very large, successful companies.

Microsoft has been operating below its Optimum Profit level for a time - ignoring this ecomonic law will lead to financial disaster. The only question is "how long before they notice and correct?"

Google likewise has to attend to this same problem. It's in a better position - under-spending - but by how much can they afford to under-perform?

---

References for "Death by Success"

I haven't found a book that documents the causes of the 2007/8 Global Financial Crisis in these terms. The financial manipulation at the root - knowingly making bad loans (sub-prime mortgages) and passing them off to others - is obviously a Ponzi or Pyramid scheme if phrased this way. Kept small, it would've been a blip... Allowed to become mega-successful, it has come close to bringing down the global financial system

Jarrod Diamonds' book, "Collapse: How Societies Choose to Fail or Succeed"
documents this for societies.

Sydney Finkelstein, in "Why Smart Executives Fail: And What You Can Learn from Their Mistakes" does it for companies and their CEO's.

Jerry Weinberg teaches the rule at the Project/Team level in simulations within "Problem Solving Leadership" and is probably in his extensive writing. "Becoming a Technical Leader: An Organic Problem-Solving Approach" is the PSL textbook.

There's enough media coverage of self-destructing celebrities, sports stars and the newly rich to illustrate the idea.
Waldroop and Butler's "The 12 Bad Habits That Hold Good People Back: Overcoming the Behavior Patterns That Keep You From Getting Ahead" makes a good start for self study.

Microsoft Troubles - V

More on the theme of "Microsoft will experience a Financial Pothole" from a Financial perspective.

For less rigorous commentaries, there's Motley Fools piece "The Two Words Bill Gates Doesn't Want You to Hear" (A: Cloud Computing) [Article requires signing-up, but can be found from Your Favourite Search Engine]
A commentary on that piece from Carbonite, a 'Cloud' vendor.

Victor Cook at Customers and Capital has a series of pieces on Microsoft and its fundamentals.
His series on 'Brands' compares GOOG & MSFT at times.
The first post in the 'Blue Ocean Strategy' series, the second, third and fourth posts.
His analyses of a number of issues are informed and enlightening - MSFT vs GOOG, the YHOO & Double-Click acquisitions...

Cook points at a March-19 2009 piece at 'The Wild Investor' called "The State of Microsoft" that starts:

Well we are not in the 1990’s anymore, and unless you plan to hold the stock for 50 years there is really no point to holding shares of Microsoft. Here is why…

and ends with:

The bottom line is that the luster behind Microsot is no longer there. Sure, there is upside to the stock, but how much. We are in a new time where the old blue chips are no longer blue. Stocks like Cisco (CSCO), IBM (IBM), and General Electric (GE) are no longer fun or smart companies to invest in.

How "The Internet Changes Everything" for Journalists

Update 1: 26-May-2009
A very kind person @ ABC took the time to read this piece and to give me some valuable comments:

• We are paid to make editorial decisions ... This is our job.
• news is about immediate events and happenings and it is short, brief and factual.
• 'News' is just one kind of ABC broadcast.
  Don't confuse it with our entire output.
• As to your specific proposals...
  - SPAM is a "dropdead" problem (my words)
  - automatically generated responses are a waste of time... listener input should be read and acknowledged by a real person,
  - but the problem is that we are all under time pressure.
  

So where to from here?
Appears to be end-of-the-road for this approach.

---

How does "The Internet Changes Everything" apply to News and Journalism at the ABC - Australia's national broadcaster?

Contents: Background, How the Internet Changes Everything and Proposal

What are the sources of Good Stories?
Do Journalists have a monopoly on sources and Perfect Judgement on story 'size' & importance?
Obviously not.

But how do folk "out of the loop" gain access to the Gatekeepers of the public media?

Consider two cases and what, if anything, has changed now if they were to be repeated:

• The "Erin Brokovich effect": through persistent talking & listening to ordinary householders, an apparently minor legal matter became massive. The romantic story of the film has the primary evidence, "the smoking gun", only arriving accidentally, and
• "Dr. Death" allegations at Rockhampton hospital: a set of nurses at the hospital attempted, for years, to raise their concerns internally & externally without getting any 'traction'. Meanwhile, people were needlessly being harmed, there's no media coverage and hence no political interest in investigating the claims.
  

Background

The ABC News website is clear, well structured, informative and completely useless and frustrating for someone like me who's not sending Press Releases, submitting news tips/vision, already known to journalists or a 'recognised industry expert'.

I spent a week trying to contact any journalist inside the ABC who might be able to pick up a small but important question that goes to the heart of the National Broadband Network Fibre-to-the-Premises proposal of the Rudd Government: the published pricing ($5,500/house) is 2-5 times higher than both the 1995 Optus/Telstra cable TV roll-out to 2-3M houses and recent FTTP installations.

Something looks very wrong, and asking this and consequential questions of both Politicians and Telco experts/consultants would make good copy and put the ABC at the forefront of this News thread.

But I can't get through... E-mails, web-form emails and phone calls (with follow-ups) have drawn a blank. I'm not some major 'name' or consultancy so my 30+ years experience in the field and a bunch of good innovations just don't count... Which is what I have to presume, because I've heard nothing back. Not even an automatic response.

From my position outside, it appears that internally ABC News operates as a set of independent 'silos' - groups that are completely isolated from one another. If you get through to anyone, they might reject it but offer no assistance in what to try next.

The next step is "go to the top". Which would be the newly appointed "Head of News", who shows up in media releases, interviews, in the Organisation Chart [PDF], but not yet in the 'Contacts' page - which curiously has only postal and telephone contact information. That implies there isn't an automatic system to update all relevant webpages. Writing to 'webmaster' should work - but from my experience, I'm disinclined to try.

If there are permanent electronic addresses to contact people in senior roles, they are not disclosed.

Ditto for any set of 'Editors', 'News Desks' or targeted 'Correspondents'.
Is it possible internally ABC News is this chaotic & unorganised?
Is this "Wall of Granite" exposed to outsiders accidental or intentional?

"The Internet Changes Everything"

Radio is a highly personal medium: Philip Adams insight is in speaking to 'the listener'. He knows he is having a personal conversation with individuals, not a group or an 'audience' as you find in theatres and sports grounds.

People listening to radio are more likely to want to continue the conversation on-line. This is facilitated on the ABC site by web-form email, forums and even a 'Complaints' facility.

The News site even has a 'Contribute' page:

If you witness a news event, the ABC wants to hear from you. We would like you to send us your newsworthy photos, videos, audio clips or even written eyewitness accounts for consideration for use on ABC News.

Can you see the assumption in there? It's insular and iconoclastic.
We find 'the news', you listen.

There's a secondary assumption:
News is only 'events' that can be represented in sound-bites and pictures. There is no allowance for informed contributions and bigger stories.

There is a simple test:

Does the system facilitate or block major public interest stories like "Dr. Death" at Rockhampton Hospital, false 'evidence' claims as in the 'Children Overboard' affair, or problems like gross waste/misuse of public money, dereliction of duty, outrageous behaviour of public officials/politicians or endemic corruption?

It is embarrassingly insular to assume that, as a publisher/broadcaster, you always know better than the entire listening audience what is going on.

That one person, using the technology well and wisely, can access and leverage community knowledge, globally, and affect a major outcome is shown by 'PJ' (Pamela Jones) of Groklaw and her influence on the "SCO case". SCO became the final licensee of the AT&T Unix codebase and sought to leverage this into a 'tax' on Linux, a re-implementation loosely based on Unix.

In the intervening years (2003 - 2007) PJ and Groklaw can be credited with unearthing and exposing many of the flaws in SCO’s case, most notably, obtaining and publishing the 1994 settlement in the USL vs BSDi case, which had been hidden from public view and played a significant role in undermining SCO’s claims to the ownership of Unix.

PJ's efforts and collaboration with the global community were instrumental in SCO losing its case. No one company, even vendors like IBM, Novell and AT&T, and certainly no consumer, had all the information nor all requisite manpower to definitively dismiss the claims.

Lesson: One person can make a difference, if they apply themselves and the technology appropriately.

The Internet is a new thing - it's not just a faster, cheaper, better way to do the same things.
If you simple-mindedly automate existing practices, you will open the floodgates and will drown in electronic verbiage.

When the 'barriers to entry', the cost in time, energy & money, of communicating are very low then people will bombard you with messages. The sheer volume and the Signal-to-Noise ratio means the content is worthless: a small army, let alone a single person, won't be able to read everything and duplicate/irrelevant information will drown out any gems therein.

Computers also hold the key to the problem - they are Cognitive Amplifiers.
They enable one person to do the work of 10, 100 or 1,000. More quickly, more cheaply and often 'better' in important ways.

The ABC News Division employs 700 professionals.
They certainly perform very well, but are they sufficient to find and research all worthy stories locally, let alone all local interest stories occurring world-wide?

What's the price to the Organisation, the Australian Media, Government/Politics and the Australian Public of missing important stories??

There are many problems to be addressed and overcome before a useful system can emerge:

• SPAM
• Security/malware - controlling & avoiding upload/dissemination
  
• Denial-of-Service attacks and webpage and other information hacking,
• Retaliatory attacks and deliberate mis-information by 'sources'.
  
• Mischief makers, Gossips and Defamatory statements.
  
• 'Personal Agendas' and Vendettas/Disgruntled persons,
• Copyright violation and Plagiarism,
• Nuisance, time-wasting, 'serial pests' and Vexatious persons

Ain't no bed of roses...

There are always going to be people who wish to remain anonymous - either completely or in the usual 'off the record' sense where they do not wish to be publicly quoted, but are willing provide a written statement and, if necessary, to stand by their comments in court.

One key technical method to address many of these problems is strong identification of posters.
This has to be of similar strength to X.509 client certificates for browsers with the concomitant off-line checks issuing them.
Off-line confirmation identity is essential - like checking the whitepages and calling the person, or sending an SMS - up to sighting 'photo id'.

A Proposal:

Immediately, three improvements would help address my frustrations:

• publish role-based, not personal, email addresses for both senior positions and the various news desks/editors/specialist correspondents that must exist internally.
• Add a new contact form for story requests, useful information and leads.
  The sorts of things needed for investigative journalists. It must include topic categorisation for automatic distribution to the news desks/story areas.
  
• Automatically acknowledge all contacts - by via email or SMS or other simple means.
  

A backend system is needed to automatically store and distribute input. Something of this sort must already exist - it certainly is there for published stories.

Writing is a skill that must be practised.

Members of the public do not have journalistic skills, nor a sense of what journalists consider 'newsworthy', nor what is needed to successfully pitch a story - even if they have found the right person. The limited feedback I've received amounts to "just write clearly". I know how to do that in a number of domains, but have no idea what journalists want and need.

This can be addressed at 3 levels:

• An on-line tutorial and example system.
  Including some template questions and suggestions for ways to both condense/summarise your information and to self-assess its 'importance' and 'newsworthiness'.
  
• A limited (5 minutes) response by a journalist to any specific questions or advice.
  
• The ability to pay for editorial help in constructing a pitch and a even a story.
  The rate would have to be $50-$75/hour for cost recovery, more to act as brake on overuse.
  

One very strong asset of the ABC is its 'Friends'.

With the increasing numbers of retired Baby Boomers - including ex-journalists, there should be a large pool of free labour available to review, categorise and respond to the information fire hose that would be unleashed.

The Internet means people can volunteer for an organisation without the classic problems of desk-space, real-time supervision, insurance and other entitlements. Volunteers work from home, when and for as long as they like. They could even be self-administering.

At the end of the day, it comes down to just one question:

What does the ABC consider its own and the community's on-going roles, and
how will it stay current with sociological, cultural & technological changes?

Alan Kay - History and Revolution in I.T.

Alan Kay invented Object Oriented Programming around 40 years ago with Smalltalk.
Kay not only has a lot to say, his accomplishments lend him credibility. In 2003 he said:

"our field is a field that's the next great 500-year idea after the printing press"

The ACM awarded him its highest honour, The Turing Award, in 2003. The short Citation:

For pioneering many of the ideas at the root of contemporary object-oriented programming languages, leading the team that developed Smalltalk, and for fundamental contributions to personal computing.

Video of his 60min talk on the ACM site, and elsewhere, a transcript. The slides & demo used are not available.

This 1982 talk for "Creative Think" brought this reporter reaction (the link is worth reading for the list of line-liners alone):

Alan's speech was revelatory and was perhaps the most inspiring talk that I ever attended.

This is my current favourite quote of Alan Kay's from Wikiquote:

"Point of view is worth 80 IQ point"

A 2004 conversation with Kay on the (deep) History of Computing Languages is well worth reading. Here are two interesting remarks:

One could actually argue—as I sometimes do—that the success of commercial personal computing and operating systems has actually led to a considerable retrogression in many, many respects. (starting circa 1984)

and

Just as an aside, to give you an interesting benchmark—on roughly the same system, roughly optimized the same way, a benchmark from 1979 at Xerox PARC runs only 50 times faster today.
Moore’s law has given us somewhere between 40,000 and 60,000 times improvement in that time.
So there’s approximately a factor of 1,000 in efficiency that has been lost by bad CPU architectures.

Kay, in this piece, also mentions a theme - the central problem of writing large systems is Scaling - the Design & Architecture of systems. Anybody can take lumber, hammer, saw, nails and produce some version of a dog-house. To scale up to something very large requires skill, discipline and insight - Architecture is literally "the science of arches", the difference between Chartres Cathedral and the Parthenon. Both contain around the same amount of material, the cathedral encloses ~20 times the volume and towers are 10+ times higher.

AK Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves.

SF The analogy is even better because there are the hidden chambers that nobody can understand.

AK I would compare the Smalltalk stuff that we did in the ’70s with something like a Gothic cathedral. We had two ideas, really. One of them we got from Lisp: late binding. The other one was the idea of objects. Those gave us something a little bit like the arch, so we were able to make complex, seemingly large structures out of very little material, but I wouldn’t put us much past the engineering of 1,000 years ago.

If you look at [Doug] Engelbart’s demo (wikipedia) [a live online hypermedia demonstration of the pioneering work that Engelbart’s group had been doing at Stanford Research Institute, presented at the 1968 Fall Joint Computer Conference], then you see many more ideas about how to boost the collective IQ of groups and help them to work together than you see in the commercial systems today.

Other interesting pages on Alan Kay talks:

• Summary of OOPSLA 97 keynote address and 1Hr streaming video on google or you can download an MP4 from that page
  
• O'Reilly interview, 2003, Daddy, Are We There Yet?
• 2003 e-mail exchange with Stefan Ram, on OOP
  
• Bill Clementson, 2006, "The Most Important Idea in Computer Science"
• Phil Windley reports on the 2006 "Organik Lectures" at U of Utah, ZDnet, "Is Computer Science an Oxymoron?", "The $100 laptop and other powerful ideas"

Relevance to Open Source and Paradigm shifts

Kay claims that 95% of people are 'instrumental reasoners' and the remaining 5% 'are interested in ideas'.

an instrumental reasoner is a person who judges any new tool or idea by how well that tool or idea contributes to his or her current goal.

He goes onto talk about reward/motivation and says that 85% of people are 'outer motivated' versus 15% 'inner motivated'.

Most people (~80%) fall into the 'outer motivated instrumental reasoners' group.
These people won't pick up an idea if other people aren't doing it. Which seems like a very wise evolutionary group tactic - if a little safe.

Kay, in his ACM talk, uses a contagion or forest fire model to demonstrate/claim that around 66% of a population is needed to achieve 'ignition'. To hit the tipping point where 'everyone is doing it' and the new idea takes over.

Kay also makes the observation that Big Ideas (he cites Unix) often take around 30 years to hit the streets - to become normally used.
And also wondered why after ~40 years Ivan Sutherland's ideas from his 'sketch' program from his ~1961 thesis haven't broken through.

Applying to Open Source Propagation

Putting this idea, if correct, of 'tipping point' to work in spreading FOSS :

• find/choose communities who are high in 'interested in ideas' (artists & creatives?)
• find a small community and intensively sell/lobby/influence it to get to the tipping point.
• leverage these communities or even artificial environments by introducing 'outsiders' (high schoolers?) to a converted environment.

But what Kay doesn't go near is that even the best marketers, PR and media people can't predict the next fad/craze/mene - they happen because they happen.

Reactionary or 'Frothing at the Mouth'?

Is my opinion "Forget the Best, Embrace the Rest" over the top, reactionary and irrelevant nonsense?

The State of Practice is beyond criticism - because there is no useful information on it.

Here are 3 questions to ask of Management theory & thought:

• Exactly why "Management is Hard"?
  
  
• What are the tasks of Management?
  i.e. a formal & unequivocal model of the dimensions of action and decision, resources & information required and skills/capability required of individuals and teams.
  
  
• How to quantify the performance of individual Managers and the Management Team?

These are fundamental questions and should at least be definitively outlined in any introductory text or course - but aren't.

Which begs the question: Why aren't they addressed?

Either I'm completely off-track & uniformed or have outlined something of merit.

If this viewpoint is of merit, What then?

Forget the best, embrace the rest.

It appears to me that 'Homo Corporatus' (the 'management classes') rejects, seemingly actively, the need for maintaining "Lessons Learned" and adopting in practice the best theories & principles known. The Operant Methodology seems to be:

Forget the best, embrace the rest.

This isn't a little or accidental. It's endemic and universal.

Look to General Management:
Has day-to-day management improved in the last century or two?
What about over the last two millennia? Did the Roman Empire/Army do much worse?

The real changes, like OH&S, Superannuation (Pensions), Recreation & Long-Service Leave, Overtime and Penalties, and 40 hour weeks, are those that are enforced by law.
The same observations apply to Project Management, especially within my field I.T. and I.S, but without any legal backing.

Can you name just two proven, established Management theories or systems that have become entrenched and established and are practised universally? If you can, I'm really interested to know.

Either "Management doesn't Matter", nobody cares how effectively/well organisations are managed or nobody with the influence or power create lasting changes cares...
None of which should be true.

In my working life there have been so many management fads, fashions and enthusiasms that I can't count or remember them. What they all have in common is their passing...

Implementing each of these fads caused a lot of upheaval, heartache and hard work. Their failure/passing wasn't due to incompetence, undermining or ignorance by the workface grunts like myself. Something larger and more subtle is at work.

Carol Kennedy's "Guide to the Management Gurus", circa 1991 & updated in 2007, describes the ideas of 33 (now ~45) 'Leading Management Thinkers'. The shame in reading the book is how simple, good & correct these ideas are, and their absence from current practice.

Which gives us a clue to the nature and solution of the problem:

Just what is the state of current practice?
There can be no problem if nothing is measured and nobody is looking.

Managers are people first and as individuals at work, just as in parenting, they are doing the best they can.

This is a systemic problem and needs a Systems Solution.
They are 'System Accidents' in Charles Perrow's parlance. ("Normal Accidents: Living with High-Risk Technologies", 1979 & 1999).
Or James T. Reason would say 'Organisational Accidents' ("Human Error" and "Managing the Risks of Organizational Accidents").

Why are people seemingly oblivious to this 'elephant in the room'?
or: "Why is this so??"

My conclusions from these observations:

It isn't an accident that world-changing work has been discarded, almost as if systematically.

Proof it doesn't have to be this way.

The Aviation Industry proves that Lessons can be identified and permanently learned across a global industry. 'All' that's needed is the will and insight to do so.

As an aside, the field of Air-Crash Investigation started in the UK in response to the 1954 crash of BOAC Flight 781, an early jet, the 'Comet'. Sir Arnold Hall was appointed to lead the investigation and established protocols for aircraft accident investigation which previously did not exist. Things could've been different, but he gave the industry a great start.

As air crashes reduce in frequency, both aircrew and investigators will become less experienced in dealing with real situations, as distinct from simulations and training drills. "Death by Success" is the other end of the spectrum. One we won't face in Aviation for quite some time.

Wikipedia Links:
BOAC Flight 781
De Havilland 'Comet'
"Seconds_From_Disaster" (TV series)
List of accidents and incidents on commercial airliners

Taylor and 'Scientific Management'

Kennedy covers the early "gurus" Henri Fayol, Max Weber and Alfred P. Sloan as well as Taylor.
I'm most familiar with Taylor and will write about his work.

Fredrick Winslow Taylor wrote "Scientific Management" reporting on tools and techniques discovered and refined around 120 years ago. He was interested in making business process efficient - and in proving his methods worked.

That work got nicely packaged in modern times as "Taylorism" which is known to be 'bad'.
Only there are four problems in that statement:

• The usual received wisdom about Taylors work is not just far from the truth, but the exact opposite of his viewpoint. Very few people have bothered to read his book.
  
• The concepts ascribed to 'Taylorism' are really 'Time and Motion' studies, the work of Frank Gilbreth ("Cheaper by the Dozen"). Although Taylor (USA) taught and knew Gilbreth (UK) their work and focus is different, though overlapping.
  
• Taylors' book contains a brace of case-studies - what he did and wrote about worked and was proven. One of his studies is a cautionary tale of a massive failure by others attempting to implement his 'Scientific Management' method. They consulted him and were told, "go slow, people have to buy into change or it will fail". They ignored his advice and failed exactly as predicted.
• Taylors' work was 'Business Process Re-Engineering' around 100 years earlier. But with the added understanding that all problems start as people problems and Change Management is necessary for any successful 'business process' change. Something Michael Hammer failed to appreciate and learnt the hard way...

Cybernetics and 'Operations Research'

After reading a 1971 popular book, "Cybernetics", it occurred to me that Control Theory and Operations Research weren't taught these days in IT & IS courses - a great loss. These theories offer powerful insight, tools and practices. To quote Alan Kay:

Perspective/Point of view is worth 80 IQ points.

An acquaintance is involved in formally assessing Academic Output at a University.
This raises a important, related, problem:

Important, even landmark, academic work, techniques and lessons are not carried forward.

This seems to me caused by a Systemic bias or a skewed reward system... or alternatively due to a lack intent by 'Management'.

The very people who's whole technique and Raison d'être is advancing the sum of Human Knowledge ('standing on the shoulders of Giants') - don't apply their own principles, tools, techniques to their profession. Let alone rigorously or with discipline - phrases oft used by them when criticising others.

If Academics fail so badly, what hope the rest of us?
Something deep, universal and subtle is going on that even they haven't noticed.

As an aside, almost all academics and many scientists are publicly financed. Taxpayers fund most Universities and much research.

How is it that the structures that are supposed to provide Good Governance and implement Public Accountability have never noticed this fact? Across many countries, cultures and systems?

This is a multiple systems failure:

• The profession itself has failed to notice a massive failing and the consequent outrageous waste of public monies.
  
• The 'checks and balances' in the political system ensuring prudent, efficient, effective use of public monies have failed.
• There appear to be no channels to raise issues such as this within public service bureaucracies, to the Auditors, into Parliament or to the Executive.
  

I.T. and Computing as 'Cognitive Amplifiers'

The notion of computers/IT as Cognitive Amplifiers is especially useful to the quintessential Knowledge Workers, Academics/Researchers, and is not new. The 1971 Cybernetics book even uses the phrase "amplify abilities" in describing what computers & systems do.

Here's some of the history:

Turing

In 1950, Alan Turing considered the question, "Can machines think?"
This is central to the notion that machines (IT/computers) can act as Cognitive Amplifiers.

abelard: copy of ACM paper by Turing: "Can Machines think?"

Vannevar Bush

In 1945 Vannevar Bush, the inventor of Hypertext, designed a system, "The Memex", to augment/amplify a researchers ability.

Key to its use and value was the ability to share 'Associative trails'. It allowed "researchers" to identify and exchange exact lines of research - all the material in one place.

wikipedia on Vannevar Bushs' "Memex"

Englebart

In 1968, Doug Englebart of Stanford's "Centre for Augmenting Human Intellect" demonstrated a working system (NLS).

It had multiple features that most systems would find difficult or impossible to implement today. Why has the State of the Art retreated?

Stanford: Englebarts "Mother of All Demos" - 1968 NLS videos

Stafford Beer

Norbert Wiener founded and named the field. Beer was a high-profile academic, author and practitioner who attempted to apply the principles and ideas.

In the early 1970's Stafford Beer in "Project Cybersyn" developed a real-time display & planning system for the whole Chilean economy. A coup put an end to it.

Such a system developed and applied to the US economy should've flagged in 2006 the upcoming problems with "Sub Prime Mortgages" and the ensuing global financial meltdown.

So why has nobody attempted to replicate his work?

wikipedia on Beers' "Project Cybersyn"

Bill Phillips

By 1949, Bill Phillips (a kiwi) at London School of Economics (LSE) built an analog computer model of the macroeconomic workings of an economy. This was used to dynamically model money flow. Both private and public sectors acquired machines. The Ford Motor Company used it to model economies as part of its forecasting and strategic planning.

Phillips is a well known/respected economist, discovering 'the Phillips Curve' linking interest rates and unemployment.

Phillips reasoning was that the dynamics of economics systems are not obvious.
For teaching, seeing the model in operation is a powerful learning tool.
But nobody does this today, as far as I'm aware with my limited knowledge.

NZ Institute of Economic Research custodians of the Phillips Machine (MONIAC)
wikipedia on MONIAC Computer

John Lions and teaching Operating Systems from the code

From my own personal experience:
In 1977 I studied "Operating Systems" under John Lions at University of NSW.

In 1976 John had the seemingly obvious teaching insight:

Teach O/S's by students reading & working on the code of a real O/S (Unix V6).

The shame of this is that 30+ years on, it's still a remarkable and rare insight.

Despite his book being widely acclaimed and documented as fundamental to the adoption and propagation of Unix and hence Linux, the approach remains ignored by others in Academe.

wikipedia on "Lions'_Commentary_on_UNIX_6th_Edition,_with_Source_Code"

Carol Kennedy's "Gurus"

• John Adair
• H. Igor Ansoff
• Chris Argyris
• Chester Barnard
• Meredith Belbin
• Warren Bennis
• Edward de Bono
• Alfred D. Chandler
• W. Edwards Deming
• Peter Drucker
• Henri Fayol
• Mary Parker Follett
• Sumantra Ghoshal
• Gary Hamel
• Michael Hammer
• Charles Handy
• Frederick Herzberg
• Geert Hofstede
• John Humble
• Elliot Jacques
• Joseph M. Juran
• Rosabeth Moss Kanter
• Robert S. Kaplan and David P. Norton
• Manfred Kets De Vries
• Philip Kotler
• John P. Kotter
• Theodore Levitt
• Rensis Likert
• Douglas McGregor
• Abraham Maslow
• Elton W. Mayo
• Henry Mintzberg
• Kenichi Ohmae
• Richard T. Pascale
• Tom Peters and Robert H. Waterman Jnr
• Michael Porter
• Reg Revans
• Edgar H. Schein
• Richard J. Schonberger
• E. F. Schumacher
• Peter M. Senge
• Alfred P. Sloan
• F.W. Taylor
• Max Weber
• Jack Welch
  
  

Inverse Turing - Modelling Human Brain with Universal Computer

Turing showed that all computer programs could be transformed to run on a very simple 'Universal Computer'.

One of the big questions this transformation allows is: Will the program Halt?
[The Halting Problem]
This, in the guise of a program to calculate the result, was used to prove some problems are not computable.

Turing also devised a unrelated idea - the Turing Test - that was at the heart of Artifiical Intelligence (AI) research for many years: If you can't see or hear what/whom you're conversing with, can you tell the difference between a real Human and a computer program?

Turning the 'Turing Test' around: Is the Human brain built from computing elements?

This isn't quite "Is it a computer?", there are two elements the structure and the componentry from which it's built.

If the Human brain is a computing device, and if its organisation and 'programming' can be discovered, then it can represented on an electronic computer. This is an unstated idea underlying the Turing Test.

The act of discovering the organisation and connections/programming could be impossible directly:

• every human brain is unique in structure and organisation, though built from the same components (neurons) and performing the same tasks (seeing, hearing, recall, ...).
  This can be trivially shown by 'plasticity' - when a person suffers a traumatic brain injury (like a stroke or lump of steel through the head) their brain keeps functioning and often regains lost capability. Tasks normally performed in one area are taken up elsewhere.
• Because of the microscopic nature of neurons and their axon connections and the sheer volume of them, it's unlikely a non-destructive 'read' of the whole brain will be possible.

But the idea is still very useful, even if we are unlikely to ever be able to clone our brains.

Exact computer models of brain functions/sub-systems can be built, examined and experimented with.
This is not useful in AI for recreating functions like speech recognition - electronic computers work so differently to neurons/axons that it doesn't yield insight or help for a 'reimplementation'.

For disciplines like Psychology, Sociology creating models of brain function and interactions takes study to a new level - it allows precise and repeatable initial conditions and impossible experiments to performed.

Most importantly, it removes 'hand-waving' explanations and forces exactitude and direct evaluation of theoretical models.

Computer animation tools, like MASSIVE, already implement 'autonomous agents' programmed with simple human behaviour/responses. This is a perfect platform to model and test Sociology models.

Microsoft Troubles - Starting signs

Previously I've written I expect Microsoft will experience a "Financial Pothole" around 2010, this is not it, only a portent of things to come.

Microsoft reported a profit slump and job layoffs today (AP, Reuters), as did many others - Intel and e-Bay are cited.

"Apple was a standout to the upside and was among the biggest lifts on the Nasdaq after it posted quarterly profit that surpassed Wall Street's expectations on Wednesday after the closing bell."

From the Guardian :

... expects Microsoft to report sales of its Windows software for PCs and laptops to drop 3 percent from a year earlier, making it the toughest quarter in eight years.

Microsoft are cutting "up to 5000 jobs", but otherwise Ballmer doesn't foresee problems:

Our products provide great value to our customers. Our financial position is solid. We have made long-term investments that continue to pay off.

From ZDnet's Mary Jo Foley:

(for 2008Q4) client revenue was down eight percent, compared to the same quarter a year ago. OEM revenue — the money Microsoft earns by charging its PC maker partners a set amount for each copy of Windows they preload on new systems — was down $465 million, or 12 percent. OEM license units were down one percent, according to the company.

Larry Digman of ZDnet says of 'netbooks':

Embedded in Microsoft’s earnings report and restructuring news was an implicit message that’s hard to ignore: Netbooks are disruptive and wreaking havoc in the industry.

Climbing out of the Ooze, or is that GNUze?

I've written on Microsoft heading for financial trouble and mounting internet security problems - but they have an underlying cause.

Both ignore the hard-won Unix principle: Less is More.
[the joke behind the pager, 'less']

This isn't just some arbitrary style choice, it's rooted deep in Computer Science and Software Engineering:

• smaller, cleaner interfaces - many fewer bugs if bugs/LOC is static (errors = LOC squared)
  
• smaller code - potentially better understandability
• the Einstein principle: As simple as possible, but no simpler.
• Controlling system entropy - smaller code, less entropy to grow.
  
• Learning curve for new maintainers is lower - "cognitive span" of individuals not exceeded
• Learning curve for programmers lower and risk of errors of misunderstanding lower.

More is Worse in Software Engineering - in every technical dimension. Microsoft has gone on to demonstrate that it is possible to have excessive 'customer features'.

But this isn't Microsoft-bashing - they are the definitive example of poor Software Development choices - why else would they have thrown away ~25,000 man-years of effort in the Windows Longhorn 'reset'? Didn't please customers or shareholders.

GNU, the self-styled leaders of Free Software (vs Open Source), subscribe to the Microsoft model of More is Better & Excessive is Best - sometimes called bloatware.

Proof is in the Software... Huge, slow, complex.

Why this Matters

There's a simple statement I keep hearing from long-term PC users:

My old 386/486 used to boot and run a word-processor faster than what I have to use at work these days.

Not just a little, but enough for ordinary users to feel pain.

If the base hardware has increased in speed/size by a factor of 25-100 in the last 15 years, why don't the same basic tasks run like the wind? I.e. When you do an Apples and Apples comparison, why are current systems slower - as Hal Linco did when comparing a 1986 Mac-plus and a 2007 AMD dual-core (for most used functions, Mac Plus beats the 2007 AMD Athlon 64 X2 4800+: 9 tests to 8!)

This isn't just whining or mis-remembering the Good Ol' Days - it's basic Physics and Computer Science.

Power Matters

Looking at Yawarra low-power systems and remembering the US$200 Everex gPC (first released Xmas 2007) put me onto this thought-train.

The fanless AMD geocode processors @0.5Ghz consume 5W (Yawarra)
Not sure what the fan-cooled 1.5Ghz VIA processor/board on the Everex consumes - but likely 10-20W.

Intel 3Ghz processors consume 50-60W (TDP), IIRC.

There's a simple figure of merit - MIPS/Watt. It seems forgotten by system builders and coders alike...

The More is Better philosophy has relied on the increasing power of common user hardware to fuel it's excesses.

Herb Sutter's seminal 2005 piece in DDJ "The Free Lunch is Over", shows a number of things - not the least that the assumption of free performance increases fueled by Moore's Law ended in 2002.

There should be no reason we don't have fast, functional systems running on 500Mhz (0.5Ghz) CPU's. That's so much more power than available just 15 years ago, it's not funny. Even just before 2000, these systems would've been Y2K capable and competitive.

As a collective, programmers and Software Engineerings need to climb out of the Complexity Ooze - promoted and encouraged by Microsoft and GNU. If not, they will find themselves bypassed.

There's no reason for most users (email, web-surfing, word-processing) can't get by with 5W desktops or laptops.

Proof:

iPhone and Android.

Then those 3Ghz fire-breathers might actually be used on tasks needed real speed - such as games.

If the PC systems manufacturers, O/S constructors and application developers don't embrace this change, they will simply become irrelevant as ultra-low-end systems and small appliances replace the behemoths...

Read the Three Dimensions of the changing PC market I've already outlined.

Mircosoft Troubles - IV

Previously I've posted on my conjecture that Microsoft will hit turbulent financial times in 2010: Microsoft Troubles III, Microsfot Troubles II and Microsoft Financial woes in 2010

This article in CNN Money/Dow Jones Newswire cites data on some of the early effects becoming apparent.

Sales of Windows grew just 2% in the first quarter of fiscal 2009, which ended Sept. 30, 2008. In most years, Windows posts double-digit revenue growth, according to company data.

There are three dimensions on which Microsoft profits will be squeezed.
Partly on volume, but mostly on margin.
[Update 1-Mar-09: Another dimension - Moore's Law CPU-speed]

Note: I'm not predicting Mircosoft will go away - I expect their brand (or descendants) to still be available in 25-50 years. I'm only saying that MSFT (the 'stock ticker' symbol of Microsoft) will hit a financial pot-hole around 2010.

The economic fundamental is their 90+% gross margin - it is much too high to sustain. Manufacturers can only sustain 25-30% long-term.

Note 2: I'm only discussing Desktop O/S. 'Office' is more exposed.
All the other products don't have huge market share.
X-box is still loss making, as far as I know.



The Dimensions of Challenge:

Desktop O/S Substitutes:

• OS/X is the fastest way to run Windows [on high-end Macs]
• there are viable desktop alternatives - Linux & OS/X [others?]
  

=> MS market share on desktop PC's can only drop.



Total PC Sales:

• At the low end: PC substitutes are eroding sales
• 'Appliances' like: iPhone, game consoles, SMS + phone browser, blackberry, ...
• PC sales in Japan had 5-Qtrs in decline to Mar-08. Leading indicator?
• PC sales growth in mature markets was almost stalled early 2008
• only gonna get worse in Current Financial Crisis
• PC market has changed - now >50% laptops

=> whilst the Total PC sales may increase for a time, the 'market mix' means the average PC value must be in rapid decline. At the low-end, Microsoft has to discount to retain market share.


Substitutes at Bottom-End & Top-End of PC market:

• Mac is disproportionately growing market share at the high-end
• Ulta Low Cost PC's - the low-end (eg ASUS Eee etc) is taking sales or causing MSFT to offer low-priced versions.
  

=> Both Market Share and Average Unit Price are dropping. Microsoft is being outsold at both the top-end and bottom-end of the 'classic' PC market.

CPU-speed Moore's Law change:

Around the start of 2004 , the speed of CPU's stopped doubling every 18-24 months.
After 5 years, per-CPU speeds have increased from around 2Ghz to a touch over 3Ghz.
Chip makers are compensating by creating more 'cores' per chip.
Throughput notionally increases, but not compute speed.

Businesses have extended their desktop 'refresh cycle' - they are extending the life of PC's and hence buying fewer.

Very often a hardware refresh was used to roll out major O/S changes, especially if it needed increased resources.
With corporate desktop hardware not being changed very often, vendors now need to be very aware of the need to restrain O/S resources required.

This dual effect accounts in large part for why Corporates have not embraced Vista.
Another reason - absence of convincing need or benefit.

Windows 2000 was a necessary upgrade for both Y2K issues and Active Directory solved the scalability problem.
Win-XP was useful - more functionality, integration of USB and maybe 64-bit.

---

So here's how I think it plays out:

• Total PC sales drop - market saturation, 'recession' and appliances shrink global PC market,
  
• MSFT achieves lower average unit price - competition from ULCPC & appliances,
• and MSFT market share drops as profits fall.
  

Secondary effects will come into play:

• as sales & margins drop, corporate clients will squeeze MSFT for better deals.
  
• clients can threaten to 'defect' to Apple etc or even threaten to go low-end to get better deal.
  
• MSFT sales teams will scramble to make targets - 'lets do a deal' will be their mantra, especially just before End of Year.
  
• MSFT will (radically) increase prices where customers are locked in and it perceives they have no alternative.
  
• think IBM mainframes pre-1990 - IBM kept ratcheting up prices as customers fled.
  
• the more people that have MS-Office replacements on their not-a-PC, then the more people will use Office substitutes. Microsoft will feel pain on multiple fronts.
  
• Google docs effect - maybe. But only as yet another substitute.
• MSFT management will react & thrash around wildly/blindly
  • think Yahoo! merger
  • guaranteed to do 'more of the same'
  • deep discounts to buy market share, disregarding profit and cash flow (X-box)
• buy more products & attempt to take-over markets again.
  
• will throw huge teams into building ever-bigger and fancier products.
• lots of hyperbolic advertising - selling sizzle w/o steak
• The management team will be totally distracted with bizarre tactics and will lose focus on fundamentals. They will not form and execute a good strategic plan.

Tertiary Effects:

• MSFT share price will plummet on their first poor profit performance.
• MSFT will 'cut costs' - at the expense of employees, not Management bonuses.
• Staff will leave as pay & condition decrease and work pressure increases.
• Long term staffers will see the Stock Price tank and their (paper) wealth evaporate
  • become demotivated & unresponsive/unproductive.
  • massive turf wars and blame-fests will erupt and drive down output even more.
    
• Key staff will jump ship (Gates said in an interview he had 6-12 key software people)
• Multiple board & management coups will be repelled.
  
• Stock market and customers will recoil in horror, but MS-Mgt won't notice [not our fault, we're do what we know works, things will come good again as they have in the past]
• The new lemming rush of corporate clients will be finding another "we can't be fired for buying" vendor.
• Spending will peak, Revenues will tank, Massive red-ink will flow.
  
• One last gasp - give away product 'to make it up to loyal customers'?
• Senior Management changes too late, market inertia drives sales down.

The only variation I can think of - will Bill Gates come back?

I think it possible he would buy all outstanding MSFT stock when it's cheap enough and then run it again by himself for a while... But I don't know the man, I can't tell.

Gates is the world's most successful business man.
You have to admire him for that and not underestimate his abilities.

---

In Microsoft, AntiTrust (Monopolies) and Patents, I wrote:

Why Smart Executives Fail
Sydney Finelstein wrote on the Causes of failure - the sorts of mistakes that really smart, successful people make. More at the Dartmouth Tuck Business School site.

This is a great resource on how Great Companies go bad - by doing exactly the things that made them top of the heap.