Future of Big Iron servers and Expensive Databases

This article came across a list, what follows was my response.

IBM and Oracle Present Rival Chips for 'big Iron' Servers

Wonderful to see the Dinosaurs still dukeing it out.

The eDRAM from IBM for the L3 cache is a big move. Like when they figured out how to use Copper in chips and reduced power use, hence heat production significantly.

I suspect we're seeing a replay of the late 1980's demise of mainframes... Not fast, not universal, not complete, but 90+% of the business goes away, killing weak supporting businesses.

Everyone but IBM's System Z and Unisys ClearPath went away - or into emulation.
[Clearpath = emulation on Xeon of 2200 & B-series]

In my view, two forces have converged to push these high-end niche processors into irrelevance:

• Patterson's Brick Wall (2006):
• Power Wall + Memory Wall + ILP Wall = Brick Wall
• "infinite" IO/Sec and virtual-RAM with PCI-SSD. eg Fusion-IO

With cheap PCI-SSD by the Terabyte, the majority of Apps/enterprises don't need:

• Big Iron Databases
• Big Iron Storage Arrays and supporting SAN's
• Big Iron multi-chip fast uniprocessing cores.

A lot of the complexity of Big Iron DB's (like Oracle) is aimed at achieving "speed" in the face of low-performing HDD's... [slow IO/sec, not streaming throughput]

If the whole of a relational DB (tables) fits in memory (or fast Virtual memory), then doesn't the DB become very simple, modulo ACID tests and writing "commits" to persistent, high-reliability storage?

Which means we might start seeing a bunch of in-memory DB's, like NoSQL, but for normal-sized DB's (1-5Gb), not large collections.

There's an economic rule on product substitution that led to the relatively quick decline in IBM mainframe sales:

• when the capital expenditure on a substitute is less than the operational costs of the current system, barriers to adoption are removed.
• capital costs are 'sunk' and can't be recovered.
• To realise savings, you have to wait for the next upgrade or refresh cycle.
• But the conversion costs have to be factored in, and incumbent vendors take care to price upgrades, even "forklift upgrades" (complete replacement) under the total cost of moving to a new solution. [Used to advantage by Tier 1 Storage Array vendors currently].
• But Operational Costs, like maintenance charges, aren't 'sunk'.
• They are due every year.
• When a whole system is less than recurrent costs, businesses can quickly and easily justify the change.
• They write-off the CapEx for the old equipment and wheel it out the door.
• Usually "Big Iron" hardware has zero residual value, even when 2 years old.
• When the equipment is on the cusp of being obsolescent, it is worse.
• Or they have to figure out how to break the lease.
• Unless they went into debt to fund the CapEx, they can move away quickly.

Hardware maintenance fees are typically 15-20% of capital costs.
Whilst Oracle licensing costs are beyond me (I don't track them) - but are becoming a major component of Enterprise Computing costs.

How many "little" DB applications need to succeed with two low-cost ($10k) servers, 3 SATA drives each in simple RAID and 1 Fusion-IO board, run as H/A with an in-memory DB?

If organisations can build a complete, high-performance, high-availability, simple-admin solution for $20-$30k per group of DB's, they can afford to deploy them immediately based on direct maintenance savings.

---

Follow-up comment:

Intel are looking over their shoulder at becoming dinosaurs. Maybe I won't live to see it, but ARM servers could very well do in the x86_64.
Ed

And so they should for most general purpose computing.

I remember seeing John Mashey of MIPS talk in 1988 where he plotted CPU speed for each of ECL, bipolar and CMOS technologies. ECL had been overtaken by then, bipolar was due to lose the lead within a few years.
The 486 in 1991 was a complete system-on-a-chip and changed the landscape.

It answers the question "Where did all the supercomputers go?"
A: Inside Intel. [and Power and SPARC. possibly Z series]

The Intel chips seek "maximum performance" - they pull all the tricks that super-computer designs used, and its is that technology that is approaching Pattersons' "Brick Wall" [heat, memory, ILP]

And as an aside, GPU's are filling the "vector processor" niche of CDC and Cray.

ARM has pursued a very different strategy, more based around 'efficiency': MIPS/Watt

So, while I agree with you, I think the situation is nuanced.

ARM processors are obvious choices for low-power and mobile/battery devices.
Because of design simplicity (small PSU, no CPU-fan) and smaller size, they'll become more interesting for low-end PC's, especially portable devices.

There is a company, Calxeda, now producing high-density ARM boards for servers.
They are hoping to leverage MIPS/Watt for highly-parallelisable loads, like web-servers.

But I can't see anyone taking on Intel soon in the supercomputer-on-a-chip market.
It's not just servers, especially for large DB's, but workstations and 'performance' laptops.

The problem with that evolution of the market for Intel is ARM taking sales from multiple market segments. Seeing that Winders-8 will run on ARM, we might see the end of WinTel for low-end & mid-tier laptops.

As a company, can Intel survive such a radical change in demand for its major product line?
Will its work on MLC flash fill the financial void?

I've no idea how that will go.
But like you said, ARM is going to shake up even the Intel server market.

The "secret sauce" that the ARM architecture has is that it's a licensed design.
Although chip design companies might not own or be able to access chip FABs within 2 or 3 design cycles of Intel, they can produce highly optimised and use-case targeted chips.

Which Intel can't do. They are focussed on the bleeding edge of CPU performance and FAB design.

Manufacturers like Apple/A5 and Calxeda can produced ARM-based designs that can outperform Intel-based systems by an order-of-magnitude on non-MIPs metrics.

As Apple has shown, there are very big markets where raw MIPs isn't the "figure of merit" in designs.

A new inflection point? Definitive Commodity Server Organisation/Design Rules

Summary:

For the delivery of general purpose and wide-scale Compute/Internet Services there now seems to be a definitive hardware organisation for servers, typified by the E-bay "pod" contract.

For decades there have been well documented "Design Rules" for producing Silicon devices using specific technologies/fabrication techniques. This is an attempt to capture some rules for current server farms. [Update 06-Nov-11: "Design Rules" are important: Patterson in a Sept. 1995 Scientific American article notes that the adoption of a quantitative design approach in the 1980's led to an improvement in microprocessor speedup from 35%pa to 55%pa. After a decade, processors were 3 times faster than forecast.]

Commodity Servers have exactly three possible CPU configurations, based on "scale-up" factors:

• single CPU, with no coupling/coherency between App instances. e.g. pure static web-server.
• dual CPU, with moderate coupling/coherency. e.g. web-servers with dynamic content from local databases. [LAMP-style].
• multi-CPU, with high coupling/coherency. e.g. "Enterprise" databases with complex queries.

If you're not running your Applications and Databases in Virtual Machines, why not?
[Update 06-Nov-11: Because Oracle insists some feature sets must run on raw hardware. Sometimes vendors won't support your (preferred) VM solution.]

VM products are close to free and offer incontestable Admin and Management advantages, like 'teleportation' or live-migration of running instances and local storage.

There is a special non-VM case: cloned physical servers. This is how I'd run a mid-sized or large web-farm.
This requires careful design, a substantial toolset, competent Admins and a resilient Network design. Layer 4-7 switches are mandatory in this environment.

There are 3 system components of interest:

• The base Platform: CPU, RAM, motherboard, interfaces, etc
• Local high-speed persistent storage. i.e. SSD's in a RAID configuration.
• Large-scale common storage. Network attached storage with filesystem, not block-level, access.

Note that complex, expensive SAN's and their associated disk-arrays are no longer economic. Any speed advantage is dissolved by locally attached SSD's, leaving only complexity, resilience/recovery issues and price.
Consequentially, "Fibre Channel over Ethernet" with its inherent contradictions and problems, is unnecessary.

Designing individual service configurations  can be broken down into steps:

• select the appropriate CPU config per service component
• specify the size/performance of local SSD per CPU-type.
• architect the supporting network(s)
• specify common network storage elements and rate of storage consumption/growth.

Capacity Planning and Performance Analysis is mandatory in this world.

As a professional, you're looking to provide "bang-for-buck" for someone else who's writing the cheques. Over-dimensioning is as much a 'sin' as running out of capacity. Nobody ever got fired for spending just enough, hence maximising profits.

Getting it right as often as possible is the central professional engineering problem.
Followed by, limiting the impact of Faults, Failures and Errors - including under-capacity.

The quintessential advantage to professionals in developing standard, reproducible designs is the flexibility to respond to unanticipated load/demands and the speed with which new equipment can be brought on-line, and the converse, retired and removed.

Security architectures and choice of O/S + Cloud management software is outside the scope of this piece.

There are many multi-processing architectures, each best suited to particular workloads.
They are outside the scope of this piece, but locally attached GPU's are about to become standard options.
Most servers will acquire what were known as vector processors and applications using this capacity will start to become common. This trend may need their own Design Rule(s).

Different, though potentially similar design rules apply for small to mid-size Beowulf clusters, depending on their workload and cost constraints.
Large-scale or high-performance compute clusters or storage farms, such as the IBM 120 Petabyte system, need careful design by experienced specialists. With any technology, "pushing the envelope" requires special attention by the best people you have,  to even have a chance of success.

Not unsurprisingly, this organisation looks a lot like the current fad, "Cloud Computing" and the last fad, "Services Oriented Architecture".

---

Google and Amazon dominated their industry segments partly because they figured out the technical side of their business early on. They understood how to design and deploy datacentres suitable for their workload, how to manage Performance and balance Capacity and Cost.

Their "workloads", and hence server designs, are very different:

• Google serves pure web-pages, with almost no coupling/communication between servers.
• Amazon has front-end web-servers is backed by complex database systems.

Dell is now selling a range of "Cloud Servers" purportedly based on the systems they supply to large Internet companies.

---

When 'ping' fails

In Networking, the 'performance objects' are links, usually end-to-end, consisting of many elements - like ethernet segments, switches, routers/firewalls, long-distance circuits and security scanner devices, laid on top of Telco/backbone services that provide dynamic and asymmetrical routes.

The most frequently used measure is 'ping' time - ICMP “echo request” packets, often 64 bytes long.
Firewalls, routers and underlying networks filter/block classes of traffic, implement traffic & flow rules and attempt "Quality of Service". There are many common rulesets in use:

• blocking 'ping' outright for 'security reasons'. Stops trivial network scanning.
• QoS settings for different traffic classes. VoIP/SIP gets high priority, FTP traffic is low, your guess on HTTP, DNS, time (NTP) and command line access - SSH, Telnet, ...
• traffic profiling on IP type (ICMP, UDP, TCP) and packet size (vs link MTU).
• traffic prioritisation based on source/destination.
  

Real-time voice needs many small packets, would like low latency/delay, and no jitter, and can stand some packet loss or data errors. FTP relies on TCP to detect packet loss & retransmit them. It likes big packets and attempts to increase bandwidth consumed through TCP 'fast-start' etc.

The only time 'ping' is accurate is within a simple ethernet segment - no rules, no firewalls, no 'traffic engineering', no link losses, no collisions, ...
Otherwise, it's a dangerous and potentially very misleading measure.

'Time of Flight' for UDP & ICMP packets is only measurable when you control both ends of the link and can match headers. Not something most people can or want to do.

TCP packets can be accurately timed - sessions are well identified, packets can be uniquely identified and they individually acknowledged. It is possible to accurately and definitively monitor & measure round-trip times and effective throughput (raw & corrected) of links and connections at any point TCP packets are inspected - host, router, firewall, MPLS end-point, ...
I'm not aware of this being widely used in practice, but that's a lack of knowledge on my part.

This is not a tutorial in Networking or TCP/IP.
Neither am I a Networking expert. I'm demonstrating that even with my knowledge, "tools ain't tools" (meaning not all tools and methods are equal),
and that using just 2 metrics, 'bandwidth' & 'latency' to characterise links is simplistic and fraught. As professionals, we have to be able to "dig under the covers" to diagnose & fix subtle faults.

Consider the case of TCP/IP over a noisy satellite phone link, the type you buy from Intelsat for ships or remote areas. The service notionally delivers a data channel of 64kbps, but is optimised for a digital voice circuit, not IP packets. The end-end link has per-bit low latency on/off the link, long transmission delays, nil jitter and limited error-correction (forward-error-correction (FEC), no retransmit) - nice for telephony. These links also have buckets of errors - which voice, especially simple PCM, easily tolerates & can even be smoothed or interpolated out with simple equipment - which will be there to handle echo cancellation.

Say you're on a ship at the end of one of these links.
People are complaining that email 'takes forever' and they can't download web pages.
You run a ping test out to various points - and everything is Just Fine.

What next?

The most usual response is 'Blame the Victim' and declare the link to be fine and 'the problem' to be too many people sending large messages, 'clogging up the link', and too much web surfing. You might set quotas and delete large emails. That might work, or at least improve things marginally.

Radio links, especially back-to-back ones, each crossing 36,000km to geostationary satellites, are noisy.
If the BER is 1:100000 (1:10^5, under the hoped for 1:10^6) and you're using the default ethernet MTU of 1500 bytes, you'll get an error every 1-2 seconds. No worries, eh?
1500 bytes = 12,000 bits = 5.3 packets/second. Or 1 in 10 packets. Hardly noticeable.
TCP/IP has a minimum overhead of 40bytes/packet (less with Van Jacobson compression).
The data payload per packet is 1460 bytes for the ethernet default MTU.

Sending a 1.25Mb file (10Mbit), that's ~856 raw sent packets and 103 errors, or ~12% retransmissions. Of those 103 resends, 12 get errors as well and are resent, and 1.5 errors of those go onto a 3d round...
Or ~115 errored packets, or 14% errors on raw packets. Just a minor problem.
There's a probability of >1 error per packet, but I don't have the maths to solve for that.

The effective bandwidth (throughput) of the link, using 970 * 1500-by packets to move 1Mb in 182 seconds, is 55kpbs. Quite acceptable.

What if there's a corroded connector or tired tracking gimbal and you get a 3db change in SNR and the BER doubles? (That's a guess, not science.)
856 raw packets and 250 1st round errors, 62 2nd, 15 3d, 4 4-th, 1 5th = 333 resends. An almost 50% increase in total number of packets needed to send the file. 223 seconds and 45kbps.
Doubling the BER again (4:10^5 or 0.004%) increases the 1st round error rate to 400 packets, or 47% - 750 retransmits in 10 rounds. 301 seconds and 33.1kpbs. Half-speed.

Back to the original BER, if you were running 'jumbo frames' (9000 by) locally & these went down the link as is, you get 0.8 packets/sec and have a 72% chance of an error in a packet. One in four of the packets would get through unscathed. 140 'jumbo frames' are sent raw, 350 packets are needed with 16 rounds of retransmission.
The file takes 400 seconds at 25kbps - a hefty penalty for forgetting to configure a switch.

The problem is that packet size amplifies error rates.
A change in BER of 0.001% to 0.004%, undetectable by the human ear, halves the throughput fo TCP/IP.
Using an MTU size of 168 (128 data + 40 TCP/IP overhead) gives good performance at a BER of 1:10^4, trading 25% protocol overhead for link robustness.

'ping', using default 512 bit packets, won't detect the error.
But who'd think the MTU was a problem when standard diagnostics were reporting 'all clear'?

Summary: In difficult conditions, the BER doesn't have to drop much for link throughput to significantly degrade.

This example is about simple-minded tools and drawing incorrect conclusions.
The Latency of the link was constant, but the Effective Bandwidth, throughput, changed because of noise or link errors.

Surely that proves the Myth: Latency & Bandwidth are unrelated.

Nope, it proves that link speed and throughput bear a complex relationship to each other.

If you had been measuring TCP/IP statistics (throughput & round-trip-time) at the outbound router, or using 'ping' with MTU of 1500, you'd have seen the average latency rising as throughput dropped. All those link-layer errors & subsequent retransmits were causing packets to take longer.

But a simple low-bandwidth radio link isn't the whole story.
It's a "thin, long pipe" in some parlances.
What was special about that link was:

• no link contention, data rate was guaranteed transfer rate.
  
• synchronous data