Why IPTV won't work in Australia - ISP Volume Charging.

It's all about cost...

A phone line costs $25-$35 just to have. Add ADSL plans on top.

ADSL plan costs vary with speed and pre-paid download volume.
Reasonable TV needs around 4Mbps these days - the highest speed and cost plans.

Turnarounds

My previous post on 'Digging Out', a methodology built from experience in a number of turnarounds, can't stand alone without some justification:
What have I done to be credible?

Here's a sample taken from a version of my CV:
Completing large business critical projects on-time and on-spec. In complex political environments achieving all client outcomes without formal authority.

• ABN online registration system - Business Entry Point, DEWRSB.
  
• Y2K conversion - Goodman Fielders (Milling and Baking)
• Y2K remediation, CDM
• DFAT (ADCNET Release 3, ADCNET build/release system)
• TNT
• Unisys/Customs (EDI & Finest/Nomad)
• CSIRO - all Australian daily weather database
• Diskrom ($350k project income in a year)

ABN registrations:
The ATO paid for and ran the project. The software contractor was combative and unhelpful. The environment complex - around 6 different organisations for this one project, and another 10 projects. To get anything done required a huge amount of effort, negotiation and discussion.

The software contractor hadn't built any load monitoring and response time facilities into the system nor made any provision for capacity planning and performance analysis.

On my own recognisance, I designed and built some simple but sufficient tools, did the performance analysis - and accurately predicted the final peak load - 20 times the design load, and after diagnosing a catastrophic failure mode, designed a simple solution.

This 100% availability over 3 months was not accidental and directly contributed to 600,000 registrations of 3.3M being done on-line (around 15 times the estimate) and the site not attracting bad press like other aspects of the operation. Definitely a high-profile site.

The software contractor had to be dragged kicking and screaming all the way through the process. But I got the outcome the client needed - the site kept running with good response time on the busiest day. Some years later I analysed the performance data from the time and uncovered a few more nascent problems that we'd skirted around.

Goodman Fielders (Milling and Baking)
This was a Y2K project - they needed to migrate a legacy (Universe/Pick) application to a Y2K compliant platform and simultaneously upgrade the software and remerge all the 6 variants.

The application ran their business - ordering, tracking, accounting - the whole shooting match.
And for accounting systems, the Y2K deadline was the start of the financial year - 1-July-1999.

The work got done, I contributed two major items: deferring non-accounting conversion and moving the new single system to a properly managed facility.


DFAT (ADCNET Release 3, UNCLgate, ADCNET build/release system)
This was bizzare, I ended up sitting 20' from the desk I used when I worked on the system being replaced ('the IBM'), when it was being commissioned.

ADCNET failed and went to trial with the developer losing on all counts. It's worth reading the decision on 'austlii.edu.au' [Federal Court]. That account is certainly more detailed than the ANAO report. It was obvious in 1995 that the project could never deliver, let alone by the deadline. So I did my tasks and went my way.

To be called back again to degug and test an email gateway between the IBM and ADCNET (R2) for Unclassified messages. This was the first time I realised that being better than the incumbent staff in their own disciplines was 'a career limiting move'. Showing experienced, supposedly expert, programmers how to read basic Unix 'man' pages and act on them was a real lesson. A major problem that caused queued messages to be discarded was found and fixed by my testing - along with a bunch of the usual monitoring, administration and performance issues being solved.

I was called back again to help with the Y2K converstion of ADCNET (departmental staff were doing it). The system was over a million lines of code and the release/development environment bespoke. And required maintenance work on the dependencies/make side of the software had never been done. A few months part-time work saw all that tamed.

TNT
Went for a year as an admin. Did what I could, but they were past redemption... Bought out by the Dutch Post Office (KPN) soon after I'd arrived.
Presented a paper at a SAGE-AU conference detailing my experience with their 'technical guru' - who'd considered himself "World's Greatest Sys Admin". Google will find the paper for you.
It was so woeful, it defies description.

Unisys/Customs (EDI & Finest/Nomad)
In early 1995 was called in to replace a SysAdmin for 8 weeks on the Customs "EDI Gateway" project. The project and site were a disaster - so much so that Unisys listed it as a "World Wide Alert" - the step before they lost the customer and hit the law courts.

In two months the team stabilised the systems, going from multiple kernel 'panics' [a very bad thing] per week, 8-10 hour delays in busy hour and lost messages - to 100% uptime over 6 systems, 1-2 second turnarounds and reasonable documentation, change processes and monitoring/diagnosis tools. The Unisys managers were very appreciative of my efforts and contributions. This same sort of chaos that was evident in the 2005 Customs Cargo clearance System debacle. [The 'COMPILE' system ran on Unisys 2200 and was being replaced over a 10-year period. It was the back-end for the EDI systems I worked on.]

So much so, that I was called back for another few months to stabilise another system running ADABAS/Natural legacy applications that provided the Financial Management & Information Systems and Payroll/Personnel system. Another high-profile, critical system.

CSIRO - all Australian daily weather database
The research scientists on the project I worked for created some tools to analyse weather data - and had found a commercial partner to sell them. The partner was not entirely happy due to extended delays and many unkept promises. I'd been told that to buy the entire dataset - a Very Good Thing for the commercial partner - was not affordable, around $20,000 for the 100 datasets from the Bureau of Meteorology. When I contacted the BoM, they not only provided the index in digital form for free, but the whole daily datasets would cost around $1,750. I scammed access to another machine with the right tape drive, wrote scripts and did magic - and stayed up overnight reading the 70-odd tapes. In pre-Linux days, there was no easy way to compress the data and move it around.

The whole dataset as supplied was 10Gb raw - and I only had a 1Gb drive on my server [$3,000 for the drive!].

It took 6 weeks to fully process the data into their file format. And of course I had to invent a rational file scheme and later wrote a program to specifically scan and select datasets from the collection.

The Commercial Partner got to release the product at the ABARE 'Outlook' conference with a blaze of CSIRO publicity. Don't know what the sales were - but they were many times better.
The research scientist got a major promotion directly as a result, and I was forced to leave for having made it all possible.

Diskrom ($350k project income in a year)
In under a year I learnt SGML, hypertext and enough about handling large text databases [1991 - before the Web had arrived], took over and completed 3-4 stalled and failing projects, migrated datasets and ssytems, designed tools and file structures/naming conventions and completed the first merge of the Income Tax Assessment Act with the Butterworths commentary, speedup processing of a critical step by 2,000 times - all of which directly contributed $350,000 in revenue [apportioned to my effort] - or around 12 times my salary.

So it's natural that everybody else in the office was given a pay rise, I was told that I was technically brilliant but not worthy of a rise and one of the 'political players' was promoted to manage the group. With a number of other key technical 'resources' I left to pursue other avenues.

Diskrom was shut down just a few years later when a new chief of the AGPS (Aus. Gov. Printing Service) reviewed the contract and decided they were being ripped off. They'd provided all the infrastructure and services, with the commercial partner paying for staff and computers - and despite lucrative contracts and overseas work, never seen any return.

Basis of Quality - What I learnt in my first years at work

On the 17th of January 1972, with 70+ others, I started as a cadet at CSR. This early work experience set the stage for how I approached the rest of my life at work. I'm come to the conclusion that these early formative experiences are crucial to a persons' performance and attitude for their entire working life. Breaking these early patterns can be impossible for some.

The cadets of my intake had the privilege to have unrivaled experiential life lessons in quality, safety, team building & working, skills development and personal responsibility.

The lessons I took with me into my career:

• Quality workmanship and precision techniques
  
• Formal quality techniques and analyses
  
• Following precise analytical processes
• Managing high work loads and meeting tight deadlines
• Responsibility for high-value processes/outcomes
  
• Satisfying stringent customer requirements
  
• Respect for, coping with and managing work in dangerous
  environments - and experience in handling/responding to hazardous
  incidents.
• And doing time sheets and "filling in the paper work" as a natural part of things.
  
  

My first 2 years of full time work as a cadet chemical engineer at CSR Ltd saw me personally responsible every day for $10M’s of product – performing sugar analysis for the whole of the Australian sugar crop. At the time the price of sugar was at an all time high. Each of us played our part in process - no one person did it all, but one person could botch the work of a whole team, or even a whole days' work.

At these NATA certified laboratories, we were trained in Chemical Analysis - but also safety and quality methods - with lives and ‘real money’ at stake.

Routinely large quantities of flammable and explosive alcohols, and highly toxic and corrosive acids were used and safely disposed of.

Deadlines were tight and fixed – each day samples for 50-100,000 tonnes of raw sugar and full analysis had to be delivered with a very high degree of accuracy and certainty that same day.

Speed, precision and absolute dependability were instilled in us alongside a clear knowledge of the value and consequences of our work - and mistakes.

We were tutored in analytical techniques, trained in reading and following exactly processes, statistical analysis, fire and safety (OH&S) skills, certified first-aid and our duties and responsibilities to our clients - the sugar producers.

It was expected that "people make mistakes" - the first rule of any precise analysis is the error range (+/- ??). The system was designed consistently to produce accurate, repeatable results with a very low margins of error. Calibrated samples were fed through the process alongside multiple samples and any 'repeats' that had failed the checking process. The performance of individual analysts and groups was tracked and monitored. People were assigned to tasks suited to their particular talents - based on objective results, not presumption or basis.

We all acquired robust team working skills. Along with how to do the boring things like time sheets with exactitude.

The next year I spent working as an Analyst in the Pyrmont Sugar Refinery.

Lots of routine and tight deadlines - and the same attention to detail, importance of results and a necessity to understand 'the big picture'.

There'd been an analyst, an ex-secretary, who'd dropped her (mercury) thermometer into the 'product stream'. She hadn't realised it was a problem, and a few hundred tons of sugar had to be thrown away and the process shutdown for cleaning. A tad expensive - many more times her wage and the cost of training and supervision to prevent.

My routine work led to uncovering a systematic problem with a very simple and preventable cause. We had a power station on-site - a real live coal burning, electricity and high-pressure steam (50 atmosphere?) producing plant. It used enough coal that when a fire in the furnace hopper caused a malfunction, the whole of the Sydney CBD was covered in a thick pall of black smoke - which made the TV news.

The plant feed-water was supposed to be closely maintained near a neutral pH - not acidic. A test station was on the factory floor, with a reagent bottle next to it. The reagent ran out, so I made a new batch - only to find that the measurements were now grossly wrong. The bottle was stored without a lid and slowly evaporated and concentrated. Over a couple of years, the pH measurement had slowly drifted and the water pushed way out of spec.

Serendipitously a major explosion or equipment failure was avoided. Replacement of the power-station, and shutting down the whole of the Pyrmont facility dependent on it for a couple of years, would've seriously impacted the whole company.

Digging Out - Turning around challenged Technical Projects/Environments

Something I wrote in 2002:

‘Digging Out’ - 7 Steps to regaining control

This is a process to regain administrative control of a set of systems. It can be practised alone or by groups and does not require explicit management approval, although that will help.

‘Entropy’ is the constant enemy of good systems administration – if it has blown out of control, steps must be taken to address it and regain control. The nature of systems administration is that there is always more than can be done, so deciding what not to do, where to stop, becomes critical in managing work loads. The approach is to ‘work smarter, not harder’. Administrators must have sufficient research, thinking & analysis time to achieve this – about 20% ‘free time’ is a good target.

This process is based on good troubleshooting technique, the project management method (plan, schedule, control) and the quality cycle (measure, analyse, act, review).

The big difference from normal deadline based project management is the task focus, not time. Tasks will take whatever time can be spared from the usual run of crises and ‘urgent’ requests until the entropy is under (enough) control.

Recognition

Do you have a problem? Are you unable to complete your administration tasks to your satisfaction within a reasonable work week? Most importantly, do you feel increasing pressure to perform, ‘stressed’?

Gather

The Quality Cycle first step is ‘Measure’. First you have to consciously capture all the things that 1) you would like to do to make your life easier and 2) take up good chunks of your time.

The important thing is to recognise and capture real data. As the foundation, this step requires consistent, focussed attention and discipline.

The method of data capture is unimportant. Whatever works for the individual and fits naturally in their work cycle – it must NOT take significant extra time or effort.

Analyse

Group, Rewrite, Prioritise.

Create a ‘hard’ list of specific tasks that can be implemented as mini projects that can be self managed. Individual tasks must be achievable in reasonable time – such as 1-2 days effort. Remember you are already overloaded and less than fully productive from accumulated over stress.

Order the list by 1) business impact and 2) Daily Work-time gained.

The initial priority is to gain some ‘freeboard’ – time to plan, organise and anticipate, not just react.

Prioritisation can be done alone if there is not explicit management interest.

It will surprise you what management are prepared to let slide – this can save you considerable time and angst.

Act

Having chosen your first target, create time to achieve it. This requires discipline and focus. Every day you will have to purposefully make time to progress your goal. This means for a short period spending more time working or postponing

Do not choose large single projects initially, break them into small sub projects.

When you start, schedule both regular reviews and a ‘drop-dead’ review meeting – a time by which if you haven’t made appreciable progress on your task to review

Review

How did it go? Did you achieve what you wanted? Importantly, have you uncovered additional tasks? Are some tasks you’ve identified not necessary.

If your managers are involved, regular meetings to summarise and report on progress and obstacles will keep both you and them focussed and motivated.

‘Lightweight’, low time-impact processes are the watchword here. You are trying to regain ‘freeboard’, you do NOT need additional millstones dragging you further into the quagmire.

Iterate

Choose what to do next. If you’ve identified extra or unnecessary work items, re-analyse.

When do you stop this emergency management mode? When you’ve gained enough freeboard to work effectively.

A short time after the systems are back in control and you are working (close to) normal hours, you should consider scheduling a break. You’ve been overworking for some time and have lost motivation and effectiveness. A break should help you freshen up, gain some perspective and generate ideas for what to do next.

Maintain

What are you and your managers going to do to keep on top of things? How did you slide into the ‘tar pit’ in the first place? What measures or indicators are available to warn if this repeats.

How will you prevent continuous overload from recurring?

Commercial Software is Good - because you Have Someone To Sue.

A friend came back from a ITIL Practitioners course with an interesting story:

• The course was mostly about the complexities of handling Commercial Licenses, no two of which are the same.
  
• The course provider made the not unexpected statement about Open Source:
  "Don't use it because you have nobody to sue".
  And they went onto ask "Why use OSS?"

His response was: "Because it's best of breed, especially rock-solid utilities & tools".
And they continued to not listen...

This note is NOT about that particular mindset [Risk Avoidance, not Risk Management].

I'd like to give him, and other technical people like him, a "slam dunk" one-liner response to each of these questions:

• Why OSS - because it's best of breed!
  Why use a bug-ridden, poor functioning piece of commercial software when the best there is is rock-solid, secure & Free and Open?
  Not only do you remove the need to sue *anybody*, you get the best tool for the job and know it will never be orphaned, withdrawn or torpedoed.
  
  Or you may be held to ransom with enormous support costs - the Computer Associates model of buying 'mature' software and raising the support costs to turn a profit until the customer base bails out.
  

• Using rock-solid OSS apps. means you are unlikely to need to sue anybody. It "just works", not "works just".
  And if you have concerns over "prudent commercial Risk Management",just hire an OSS support organisation who's got both "Professional Indemnity" and OSRM insurance.
  
  

And I tried to quickly find *two* significant lists for him:

• widely used open-source software [Apache, Samba, Perl-PHP-Python-Ruby, gcc/make/cvs/subvers, Eclipse, ...]
  
  The caveat on this list, is that I need estimates of the market share or extent of use of the software. Viz: For apache, the netcraft survey:
  
  
• OSS support organisations. [remember Linuxare ?]

If you have pointers to that data, I've love to hear from you.

Who can we sue? Or - the Myth of Riskless I.T. Management

This started as a conversation on an Open Source list - on how to respond to people that assert:

"We can't use Open Source because there's Nobody to Sue".

Why "IT Service Delivery and Management" should be an Academic Discipline

IT Service Delivery is where "the rubber hits the road". Without sufficiently capable and reliable IT infrastructure every other piece of the IT equation - Architecture and Design, Business Analysis, Project Management, Software Engineering, Software Maintenance, Information Management, Data Modelling. ... - becomes irrelevant. All other effort is wasted is the services don't run for the users.



All the potential benefits of IT, the 'cognitive amplifier' effects and leveraging people's skills and experience, rely on the delivery of IT Services.



Where is the academic discipline that:

• Defines IT Service Delivery (and it's OAM components - Operations, Administration and Maintenance)
• Provides a framework to compare and audit the performance of IT Service Delivery in an organisation to benchmarks relative for the industry.
  
  
• Defines absolute and relative performance of individuals, teams and IT organisations.
• Defines and explores performance, effectiveness, utilisation and 'service benefit conversion' metrics?
  
  

If the goal of IT/IS is to "deliver a business benefit" - but the benefits aren't trivially measurable, the deep knowledge/experience of the discipline of Marketing can be brought to bear. The first step in every project is to define 'the desired benefit', how it will be measured and reported, and the break-even or cancel point.



The academic discipline that informs practitioners, management and the profession on how to actually realise the benefits of IT/IS systems in practice.

ITIL and ISO 20,000

"ITIL" (IT Infrastructure Library) was first formulated in 1989 by the UK OGC (Office of Govt. Computing) to provide a common language and framework or conceptual model for IT Operations (now 'service management') - necessary for the process of tendering and outsourcing.



In 1999 it was re-released ('version 2') and new books written.

Mid-2007 sees the release of 'version 3' - another rethink and rewrite.



In 2000 ITIL spawned a British Standard, BS15000, revised and updated in 2002. In Dec 2005 BS15000 was adopted internationally as IEC/ISO20,000. It sits alongside "Information Security Management" ISO17799 [formerly BS7799:2002] and "Information Security" ISO27001. BS25999 address "Business Continuity".



Forrester Research in early 2007 reports (in "CIOs: Reduce Cost By Scoring Applications" by Phil Murphy) that 'IT Service Delivery' (Forrester calls it "lights on" operations and maintenance) is accounting for a rising percentage of IT budgets. Referenced in "Maintenance Pointers".

Reasons for a discipline of IT Service Delivery and Management

• The Forrester survey of October 2006 reports IT Service Delivery consumes 80% of more of IT budgets - up from 60-65% ten years ago.
• 100% of the User Utilisation of IT Software, Systems and Services is mediated by Service Delivery. It's where "the rubber hits the road".
  
  
• IT is useful in business because it's a cognitive amplifier - it amplifies the amount of useful work that people can perform/process. IT provides "cheaper, better, faster, more, consistent/correct".
• Business and Government are now dependent on their IT. We've crossed the event horizon where [in the 'developed' world] it's possible to resile from IT systems.
• IT is arguably still the greatest single point of leverage [staff effectiveness amplifier] available to organisations.
• Service Delivery is the anchor point for "What Value does IT Deliver?"
  
  

Where the 'IT Services' discipline belongs

There are two requirements for a faculty teaching 'IT Services' or 'IT Service Management'

• Business and Management focus, and
• Ready access to large, complex "IT Services" installations

Traditional computing and IT faculties are focussed on the internal technical aspects of computing. 'IT Services and Management' is about delivering and realising Business Benefits - the managerial focus. The necessary disciplines and knowledge/expertise already exist in Business/Commerce/Management Schools - and are somewhat foreign to traditional CS/ISE/IT Schools.



Canberra, accounting for 20% of the IT expenditure in Australia, is well placed to initiate 'IT Service Delivery and Management' in this country.

Flash Memory, Disk and A New Storage Organisation

The raw data for this table. It's not definitive, but meant to be close to reality. I haven't included tape media, because I have no reliable price data - and they are not relevant to the domestic market - CD's and DVD's are one of the best, cheapest and most portable/future proof technologies for enterprise and domestic archives and backups.
A Previous Post quotes Robin Harris of ZDnet (Storage Mojo).

Edit (04-jun-97): George Santayanda on his storage sanity blog writes on the Flashdance. Cites 80%/pa reduction in flash prices, break-even wit HDD on 2010/11. He's been in storage for years and is a senior manager. And doesn't take things too seriously.

And he points to a Powerpoint by Jim Gray on the Flash/Solid State disk. Worth the read.

Storage Price Trends

The Yr/Yr ratios are used for forward projections.

The 'Est Flash' column uses the current 'best price' (MSY) for flash memory and a Yr/Yr ratio of 3.25.

Flash memory is now very much cheaper than RAM - forward projections not done.

Year	RAM $/Gb	Flash $/Gb	Est Flash $/Gb	Disk$/Gb	DVD $/Gb	Max flash	Max Disk
2002	936.00	1176.00	1176.00	3.13	0.57	256	120
2003	384.00	700.00	700.00	2.75	0.38	512	160
2004	298.00	400.00	400.00	1.29	0.26	1024	250
2005	189.00	158.00	158.00	0.82	0.17	1024	320
2006	135.00	87.50	87.50	0.62	0.17	2048	500
2007	159.00	24.75	11.06	0.37	0.15	4096	750
Yr/Yr ratio		2.26	3.25	1.57	1.33
2008		10.95	3.40	0.24	0.11
2009		4.84	1.05	0.15	0.08
2010		2.14	0.32	0.10	0.06
2011		0.95	0.10	0.06	0.05
2012		0.42	0.03	0.04	0.04

Depending on the"Year-on-Year" ratio you choose for the reduction in $/Gb of Flash memory, and if you think both flash and disk drives will continue their plunge down the price curve, solid state memory (flash) may be the cheapest form of storage in under 5 years.

New Storage Organisation

Backups and Archives

With the price of large, commodity disk drives driving down near DVD's, and probably overtaking, within 5 years - and that's ignoring the cost of optical drives and the problems of loading the data you want. Why would you not want to store backups and archives on disk?



For safe backups, the disks cannot be in the same machine, nor actually spinning. If the disk is in a USB enclosure, this meaning being able to spin it down by command.



Small businesses can effect a safe, effective off-site backup/archive solution by pairing with a friend and using 'rysnc' or similar over the Internet (we all have DSL now, don't we?) to a NAS appliance. The NAS does need to store the data encrypted - which could be done at source by 'rsync' (yet another option) or created by using an encrypted file system and rsync'ing the raw (encrypted) file. Best solution would be to have the disks spin-down when not being accessed.



This technique scales up to medium and large businesses, but probably using dedicated file-servers.



And the same technique - treat disks like removable tapes - applies. If drives are normally kept powered down, the normal issues of wearing out just won't arise. Rusting out might prove to be an issue - and little effects like thermal shock if powered up when very cold.

Speed, Space and Transfer Rate

Robin Harris of ZDnet etc writes on Storage. He's flagged that as commodity disks become larger, a few effects arise:

• single-parity RAID is no longer viable, especially as disk-drive failure is correlated with age. Older drives fail more often. The problem is the time to copy/recreate the data - MTTR, and the chance of another failure in that window whilst unprotected.
  
  
• Sudden drive failure is the normal mode. Electronics or power supply dies - game over.
  
  
• Common faults in a single batch/model are likely to cause drives to fail early and together, and
• RAID performance is severely impacted (halved) when rebuilding a failed drive.

Harris is fond of quoting a single metric for drives: (I/O per second) per Gb. Which is a great way to characterise the effective speed of drives. The size of drives has been doubling ever couple of years - but the speed (rotational and seek) has been increasing much slower... Big drives, even lashed together in RAID arrays, can't deliver the same effective performance as a bunch of smaller, older drives.



This single figure of merit is half the equation: The other side is "time to copy".

That scales with transfer time, size, on-board cache and sustained read/write speeds.

What the World Needs Now - a new filesystem or storage appliance

Just as disk drives are bashing up against some fundamental limits - bigger is only bigger, not faster - Flash memory is driving down in price - into the same region as disks. [And nobody knows when the limits of the magnetic recording technology will be reached - just like the 'heat death' of Moore's Law for CPU speed in early 2003.]



Flash suffers from some strong limitations:

• Not that fast - in terms of transfer rate
• Asymmetric read and write speeds (5-10:1)
• bits wear out. Not indefinite life.
• potentially affected by radiation (including cosmic rays)

But it's persistent without power, physically small, very fast 'seek time', relatively cheap per unit, simply interfaced, very portable, (seems) reliable and uses very little power. Cheap flash memory only transfers around 5Mb/sec. Sandisk "Extreme 3" Compact Flash (CF) cards targeted at professional photographers, write at 20Mb/sec (and "extreme 4" double that).



"Plan 9", the next operating system invented by the group who designed Unix (and hence Linux), approached just this problem. Files were stored on dedicated "File Servers" - not that remarkable.



Their implementation used 2 levels of cache in front of the ultimate storage (magneto-optical disk). The two levels of cache were memory and disk.



The same approach can be used today to integrate flash memory into appliances - or filesystems:

• large RAM for high-performance read caching.
  
  
• large, parallel flash memories for read buffering and write caching
• ultimate storage on disk, and
• archives/snapshots to off-line disk drives.
  
  

The disk drives in the RAID still have to have multiple parity drives, and hot spares.




The Flash memory has to be treated as a set of parallel drives - and probably with parity drive(s) as well.


This arrangement addresses the write performance issues, leverages the
faster read speed (when pushing cache to disk) and mitigates the effect
of chip failure, bits wearing out and random 'bit-flips' not detected
and corrected internally.



They only deep question is: What sort of flash drives to use?

• compact flash are IDE (ATA) devices. Same pin-out (less the 2 outside) as 2.5" drives
• SD card is small, cheap, simple - but bulk connection to computers aren't readily available
  
  
• USB flash is more expensive (more interfaces), but scales up well and interfaces are readily available.
• Or some new format - directly inserted onto PCI host cards...
  
  

USB or CF is a great place to start.

CF may cause IDE/ATA interfaces to re-emerge on motherboards - or PCI IDE-card sales to pick up.

Why Ideas are 'Cheap' or Execution is everything

"Genius is 1% Inspiration and 99% Perspiration": Thomas Alva Edison

Summary

Ideas cost very little to generate and without substantial additional effort, come to nothing. But new ideas are the only starting point for new things - so which is more important, coming up with the idea or making it concrete? Both are necessary and as important as one another - without the other, neither will lead anywhere. Criticism of others ideas without substantial evidence,proof, counter example or working demonstration is churlish. "Put up or Shut up" is a reasonable maxim for critiquing ideas. Ideas only take real form and viability if they are the subject of robust and probing debate and defence. It is better to fail early, amongst friends, than publicly and spectacularly. It's easy to confuse a profusion of ideas with "invention". The marker of "usefulness" is the follow-through from ideation to implementation.

Microsoft, AntiTrust (Monopolies) and Patents

MSFT is threatening to Sue the Free World.
Patents are a state-granted Monopoly in return for full disclosure.

• Patents are useless unless defended.
• Patents are granted in a single jurisdiction at a time - there are no 'global' patents.
• Patents are uncertain until tested in court - by the full panoply of judges, counsel and mountains of paper.
• Patent 'trolls' and 'submarining' exist (and are legal tactics) - people who play the system for Fun and Profit. They hide out until someone is successful, then don't try to license their patents - but sue (for large amounts).

Microsoft may claim that code infringes its patents - but that's just a posture. If they were for real, they'd be launching court cases to decide the matter.

Microsoft Troubles - III

Microsoft threatens to Sue The Free World.

Groklaw comments on MSFT threatening to sue "Patent Violations".

CNN/Fortune Original article (probably)

ZDnet (Mary Jo Follet)

Driving Disks into the future

Robin Harris of ZDnet "Storage mojo" has written a series of posts on factors affecting the future of Disk Storage. These are my reactions to these quotes and trends, especially in flash memory.



Flash getting "70% cheaper every year" - hence more attractive:

"Every storage form factor migration has occurred when the smaller size reached a capacity point that enabled the application, even though it cost more per megabyte."

"With flash prices dropping 70% a year and disks 45%, the trend is inexorable: flash will just get more attractive every year."

The problems with RAID and Big Drives:

"There are three general problems with RAID: Economic, Managerial, Architectural"

• RAID costs too much
• Management is based on a broken concept [LUN's]
• Parity RAID is architecturally doomed

"The big problem with parity RAID is that I/O rates are flat as capacity rises. 20 years ago a 500 MB drive could do 50 I/O per second (IOPS), or 1 IOPS for every 10 megabytes of capacity. Today, a 150 GB, 15k drive, the ne plus ultra of disk technology, is at 1 IOPS for every 750 MB of capacity. Big SATA drives are at 1 IOPS per several gigabytes. And the trend is down."

What a "Web Business" wants from Storage Vendors:

What a Web Business wants [Don MacAskill of 'smugmug']:

• External DAS for the database servers .. and dual-controller arrays [simplified recovery after server death]
• Spindle love. Typical array has 14.
• No parity RAID. RAID 1+0.
• 15k drive love. Speed is good.
• Love drive enclosures with odd numbers of drives. Makes keeping one hot spare easy.
• Love big battery-backed up write caches in write-back mode. Because super-fast writes are “. . . easily the hardest thing in a DB to scale.”
• Disable array read caching: array caches are small compared to the 32 GB of RAM in the servers. reserve all array cache for writes.
• Disable array pre-fetching: the database knows better than the array.
• Love configurable stripe and chunk sizes. 1 MB+ is good.

"Don should be the ideal array customer: fanatical about protection; lots of data; heavy workload, not afraid to spend money. Yet he isn’t completely satisfied, let alone delighted, by what’s out there. A lot of the engineering that goes into arrays is wasted on him, so he’s paying for a lot of stuff he’ll never use, like parity RAID, pre-fetch and read caching."

And 'the future of Storage'

The future of storage:

"The dominant storage workload of the 21st century. Large file sizes, bandwidth intensive, sequential reads and writes."



"(OLTP) Not going away. The industry is well supplied with kit for OLTP. It will simply be a steadily shrinking piece of the entire storage industry. OLTP will keep growing, just not as fast as big file apps."



"Disk drives: rapidly growing capacity; slowly growing IOPS. Small I/0s are costly. Big sequential I/0s are cheap. Databases have long used techniques to turn small I/Os into larger ones. With big files, you don’t have to."



"The combination of pervasive high-resolution media, consumer-driven storage needs, expensive random I/0s and cheap bandwidth point to a new style of I/O and storage. The late Jim Gray noted that everything in storage today will be in main memory in ten years. A likely corollary is that everything analog that is stored today will be digital in 10 years."

Response to Cognitive Work Load - Huh?

Another related question I've been trying to even discover the correct name for over the last 10 years.
I can't believe something so fundamental to I.T. and the knowledge economy could go unstudied.

I frame it as "Human cognitive response to workload".

There is a whole bunch of data on "human physiological response to workload" - like the US Navy and how long stokers can work at various temperatures (and humidity?).

This goes to the heart of computing/programming - being able to solve difficult problems, and managing/reducing defects/errors. In my career, I got very tired of bosses attempting to get more work done by "forced marches". 80 hour weeks aren't more productive - they just insure a very high defect rate and amazing amounts of rework.

The best I have been able to find is Dr Lisanne Bainbridge and her work on "mental load".

What I wanted to discover is:

• that for each individual there is an optimal number of 'brain work' hours per week
• the effect of physical & mental fatigue and sleep deprivation on 'brain work' output, degree of difficulty tasks and error rate.
• the recovery time for strenuous (mental) effort - working 50, 75 and 100 hours / week requires recovery, but how much?

If you, gentle reader, have any leads/pointers on this I really appreciate it :-)

Even if someone knows what the field is called or refer me to the peoplethat do know.

Teams - Where's the proof?

Addition 24-May-2007
Johanna Rotham author and consultant answered an e-mail from me.
Johanna is involved in the Jerry Weinberg and Friends AYE - Amplifying Your Effectiveness - conference. Johanna's book "Behind Closed Doors" is on this blog and highly recommended for I.T. Technical Managers. Another interest of Johanna's: Hiring the Best People.

Johanna's thoughtful response:

Part of the problem is I can't do two of the same project where one is set up as an integrated team and the other is a bunch of people who don't have integrated deliverables. I can tell you that the projects where the people are set up with committed handoffs to each other (Lewis' idea that one person can't work without the rest of them), have better project throughput (more projects per time period) than the groups of people who do not have committed handoffs to each other. But that's empirical evidence, not academic research.

Defining I.T. Service Management

Objectives (The What)

Having begun around 1950, the world of Commercial I.T. is now mature in many ways. "Fields of Work" and professional taxonomies are starting to become standardised. Professional "Best Practices" are being documented and international standards agreed in some areas.

For the first time, audits of one of the most pragmatic I.T. disciplines, "Service Management", are possible with ISO 20,000. Business managers can now get an independent , objective opinion on the state of their I.T. operations - or of their outsourcers.

Being "documented common sense", ITIL and the related ISO 20,000 are good professional guides, but not underpinned by theory. Are there any gaps in the standard? How does Service Management interface with other IT Fields of Work? and What changes in those other disciplines are necessary to support the new audited practice?

Analysis of the full impact of I.T. Service Management, creation of a full taxonomy and definitions of "I.T. Maturity" are beyond the scope of a small "single researcher" project.

Approach (The How)

ITIL Version 2 and 3 and ISO 20,000, as published documents, form the basis of the project.
Prior work in the field has yet to be identified. Secondary research will be the first step.

Each of the models will be codified and uniformly described, then a 3-way comparison performed. A Gap Analysis done of the 3 models, and a formal model built describing "I.T. Service Management" and its interfaces built and each of the existing approaches mapped to it.

Importance/Value (The Why)

The global economy, especially businesses in the "Western Industrialised World" are increasingly dependent on I.S./I.T. and their continued efficient operation. Corporate failures partially due to I.S./I.T. failure have occurred. Improving delivery of I.T. Services and the business management and use of them is important to reduce those failures in the future.

The advent of ubiquitous and universal computing requires concomitant development of business management.

There assertions are considered axioms in this context:

• Organisations these days are dependent on their I.T. Operations.
  
• I.T. cuts across all segments of current organisations.
  
• I.T. defines the business processes and hence productivity of the whole organisation.
  
• What you don't measure you can't manage and improve.
  
• Improving the effectiveness of I.T. Operations requires auditable processes.
  
• Common I.T. Audit and Reporting Standards, like the Accounting Standards, are necessary to contrast and compare the efficiency and effectiveness of I.T. Operations across different organisations or different units within a single organisation.
  

I.T. is a cognitive amplifier, it delivers "cheaper, better, faster, more, all-the-same", through the embedding of finely detailed business processes into electronic (computing) systems.

For simple, repetitive cognitive tasks, computers are 1-5,000 times cheaper than people in western countries.

From this amplification effect, computers still provide the greatest single point of leverage for organisations. The underpin the requirement to "do more with the same", improving productivity and increasing profitability.

The few studies of "IT Efficiency" that are available show that IT effectiveness is highly variable and unrelated to expenditure.

The value-add to business of a complete I.T. Service Management model is two-fold:

• manage down the input costs of the I.T. infrastructure and Operations and,
  
• audit assurance for the board and management of the continued good performance of I.T. Operations.
  

[A 1990 HBS or MIT study into "White Collar Productivity" - reported a decrease in the first decade of PC's]

Previous Work (What else)

There is much opinion in the area, without substantive evidence: e.g. Nick Carr and "Does IT Matter?" The McKinsey report/book on European Manufacturers and their I.T. expenditure versus financial performance shows there is no co-relation between effort (expenditure) and effect (financial performance).

"Commonsense" IT Practitioner approaches, SOX, ITIL and COBIT and others, do not address the measuring and managing of I.T. outputs and interfaces and their business effects, utiliation and effectiveness.

Jerrry Landsbaum's 1992 work included examples of their regular business reports - quantifiable and repeatable metrics of I.T. Operations phrased in business terms.

Hope to find (The Wherefore)

• Create a formal model for I.T. Operations and its performance within and across similar organisations.
  
• From the model, generate a standard set of I.T. performance metrics.
  
• Generate a set of useful I.T. Operations Business Impact metrics.
  

Report Outline

• Coded process models of ITIL version 2, 3 and ISO 20,000.
  
• 3-way comparison of ITIL version 2, 3 and ISO 20,000.
• Gap Analysis of ITIL version 2, 3 and ISO 20,000 models.
  
• Formal I.T. Service Management model.
  
• Common I.T. Service Management internal metrics and Business Impact
  metrics flowing from the model.
• Interfaces to other I.T. and business areas and changes necessary to support audits of I.T. Service Management.
  
• Further Work and Research Questions
  

Execution Phases

• Learn ITIL Version 2 - Service Managers Certificate course [complete]
• Learn ISO 20,000 - IT Consultants training [in process]
  
• Acquire and learn ITIL Version 3 [depends on OGC availability. mid/late 2007]
• Create/identify process codification.
• Codify ITIL version 2, 3 and ISO 20,000
• Compare and contrast coded descriptions. Report.
• Create/adapt process description calculus for formal model.
• Create formal I.T. Service Management model.
• Derive interfaces to business and other I.T. processes
• Derive internal metrics, role KPI's and business impact metrics
• Finalise report.
  

Bookshelf I

These are books on my bookshelf I'd recommend. Notes on them later.
Pick and choose as you need.

Personal Organisation

Personal Efficiency Program - Kerry Gleeson [older]
Getting Things Done - David Allen [newer]

Teams, People, Performance

Practice What you Preach - David H Maister [Numerical model relating Profitability to Staff Morale/Treatment]
How to be a Star at Work - Robert E Kelley
Team Management Systems - Margerison & McCann

Maimum Success: Breaking the 12 Bad Business Habits before they break you - Waldroop & Butler
[rereleased as] The 12 Bad Habits that hold Good People bBack

No Asshole Rule - Robert Sutton

Execution - the art of Getting things done (in big business)

Who says Elephants can't Dance - Louis V Gerstner [on Execution and 'Management is Hard']
Execution - Bossidy & Charan
Confronting Reality - Bossidy & Charan

Gallup Research

First, Break all the Rules - Buckingham & ??
Now, Discover your Strengths - Buckingham & Clifton [old]
StrengthsFinder 2.0 - Tom Rath [current]
12: The Elements of Great Managing - Wagner & Harter
The One thing you need to know - Buckingham

Off the Wall - Different ideas on Management and Leadership

Contrarian's Guide to Leadership - Steven B Sample
Simplicity - Jensen

Intelligent Leadership - Alistair Mant [old]
Maverick - Ricardo Semler. [old]
The 7-day Weekend - Ricardo Semler [new]

Wear Clean Underwear - Rhonda Abrams
Management of the Absurd - Richard Fearson
Charisma Effect - Guilfoyle

Computing Management

Measuring and Motivating Maintenance Programms - Jerry Landsbaum
any of the 50 books by Robert L. (Bob) Glass

Why Information Systems Fail - Chris Sauer
Software Failure : Management Failure - Flowers

Jerry Weinberg Prolific author - Quality, People, Teams, Inspections & Reviews, technical, ...
Quality Software Management - 4 book series
Becoming a Technical Leader
Weinberg on Writing - the Fieldstone Method
Secrets of Consulting
Psychology of Computer Programming
-- and another 40 or so --

Peopleware - DeMarco & Lister from Dorset House Publishing specialising in why people matter.

Project Retrospectives - Norm Kerth
Programming on Purpose - PJ Plauger

Mythical Man Month - Fredrick Brooks [I don't have a copy]

"IT Doesn't Matter" - Nicholas G Carr [read a synopsis, don't buy]

The End of the Internet, or the Microsoft Users Net-Meltdown?

The 2005 Australian Computer Crime and Security Survey(PDF) reports that at the end of 2004 "the hackers turned pro". The 2006 ACCSS indexACCSS index may be easier for downloads. [In 2016, the ACCSS was replaced by "the BDO and Australian Cybercrime Survey".]

For 2-3 years now, most malware has satisfied the definition of Organised Crime:

it's theft, it's purposeful, it's co-ordinated.

In an August 2006 post, I reported the ACCSS comments and new comments from SANS .

ZDNet now report that Rootkits becoming increasingly complex and operate by stealth. They say:

Rootkits -- malicious software that operates in a stealth fashion by hiding its files, processes and registry keys--have grown over the past five years from 27 components to 2,400, according to McAfee's Rootkits Part 2: A Technical Primer (PDF).

If you use a Microsoft system and connect to the Internet without extensive protection, you should be afraid, very afraid. And even large organisations who do everything right, are still open to targetted "zero day" attacks. The first Windows Vista security problems are being reported. It's better than their previous efforts, but still contains significant security flaws. The Whitehouse mandated a minimum security configuration for all US Federal Government Vista destops.

Microsoft troubles - II

Follow up to a previous post on MSFT hitting a 'financial pot hole' by 2010. The numbers look very, very bad to me. The seeming lack of management response and apparent leadership would deeply disturb me as a shareholder...
The Paul Graham piece Microsoft is Dead and the follow-up were a prompt for this post.

Startups: selecting and nuturing.

A comment on Paul Grahams post Why to Not Not Start a Startup.

Paul along with Robert T Morris (author of the 1988 Morris Worm, now MIT assoc. professor) run a Venture Capital firm.
They run Startup School as well. An exceptional idea.

At the end of this is a list of Paul's 16 points.

Web 2.1 - Meta-tags by default

Why do we need fine products like Content Keeper, when the problem is one that should be solved at source?

[11-Apr-2007 Addition]
The "Kathy Sierra" affair caused Chris Locke, co-author of Cluetrain Manifeso to post his version/take. My take from reading about the affair.

This whole affair unfolded because "Web 2.0" not just allows, but
enforces, anonymity. Provable Identities don't exist.

In an hour's scrolling through posts, I never saw this point [or anything like it] made.
How far would this thing have gone if the police could've tracked the posters quickly and unequivocally?
Presumably within a day or so the perpetrators would've been identifiedand action initiated, legal jurisdictions allowing.

There are good reasons to allow & support anonymity on the Web -"Freedom of Speech" is part of it, along with denying Political suppression and enabling 'whistleblowing'.

But the ugly human stuff of stalking, intimidation and control-by-fear need effective checks and consequences.

[End Addition]

Knowing the type of content you are downloading is a basic right - the same way that we don't go into newsagencies, bookshops and libraries and get surprised by the content. The same way that various TV stations will broadcast 'social content' warnings before some programs (violence, 'disturbing or graphic images', 'images of deceased people' and even 'images of surgery'). Our society has very well developed methods of flagging content that some audiences may wish to avoid - right up to full TV, movie & print "classification" and censorship. Plus we have blanket bans, enshrined in legislation, on things like "kiddie porn" and "snuf movies".

Simple minded banning of pages based on keywords or URL makes a priori judgements of what will and won't offend the audience - or under high-control regimes, what is or is not banned/seditious material. Then it becomes a simple "arms race" - two camps competing against one another (attack and defense), and by definition the reactive side can only respond once a new exploit/mechanism is noticed and identified. Yep, it's effective against people obeying the rules, but at the price of massive collateral damage and never being sure you're not compromised.

Generally, the USA is particularly sensitive to sexual matters, but not to violence. Sweden mostly has very different mores...
Filtering all pages that mention 'breast' or it's (English language) derivatives and colloquialisms fails in many ways, especially for medical & pregnancy issues ('false positives') and is easily circumvented by mistyping, obfuscation or using images ('false negatives') and is completely irrelevant for non-English language pages.

In the world of IT Security, this is why we now have Firewalls andIntrusion Detection Systems [and now systems that actively seek to confuse/entrap/counter attackers.] Funny - just like in the real world.

I'm thinking the web-server is the place to insert consistent meta-tags into content.
And that requires a minimum additional two publication stages - author, reviewer, editor/publisher - [as described by Peter Miller in his Aegis Documentation piece (82Kb PDF ) Aegis Is Only For Software, Isn't It?].

Nothing publicly published should go untagged - and that needs independent review and an enforced process to
[OK, so where does that leave the wonderful world of 'blogs'?]

We live in interconnected communities, now global in Cyberspace. All of us have sensitivities that should be respected and the publishing world evolved over many centuries a tradition of "no surprises". It's a convention that has served us well before Cyberspace, it would serve us there as well or better - with everyone "just one click away" from your content.

Free Speech is only a Right in some countries.
Censorship is a given and necessity, even in the most "enlightened" countries - where it might be called 'national security' :-)
And there are globally shared mores/values/injunctions against such things as child pornography and worse.

It's not an even playing field, and will never, can never, be.

My opinion is that laws like the DMCA [USA - Digital Millennium Copyright Act] and the Australian "anti-spam and pornography" laws [no refs] are wrong-headed and irrelevant at best - and counter-productive at worst.

With the Global Net and One Shared Cyberspace, and many cultures, beliefs, religions, etc etc, "Web 2.0" needs to add:
mandatory content tagging.

Then we can adibe by our tired-and-true convention "no surprises" and respect all our differences and sensitivities.