SteveJ-on-IT: CPU

Showing posts with label CPU. Show all posts

2014/03/30

The New Disruption in Computing

Since 2000, we've been progressively bumping into limits, or end-points, in all areas of silicon technologies. Some I've mentioned previously.

The thesis of this piece is the Next Technology Disruption is No Disruption, No Revolution:

instead of exponential growth of technologies, decreasing unit prices and increasing volume sales, we're now seeing zero or slow growth, steady or increased unit prices (especially if supply-chain is disrupted), and in all but a few market segments, sales are in decline and profits are stressed in many vendors. I believe these are linked.

Future of Big Iron servers and Expensive Databases

This article came across a list, what follows was my response.

IBM and Oracle Present Rival Chips for 'big Iron' Servers

Wonderful to see the Dinosaurs still dukeing it out.

The eDRAM from IBM for the L3 cache is a big move. Like when they figured out how to use Copper in chips and reduced power use, hence heat production significantly.

I suspect we're seeing a replay of the late 1980's demise of mainframes... Not fast, not universal, not complete, but 90+% of the business goes away, killing weak supporting businesses.

Everyone but IBM's System Z and Unisys ClearPath went away - or into emulation.
[Clearpath = emulation on Xeon of 2200 & B-series]

In my view, two forces have converged to push these high-end niche processors into irrelevance:

Patterson's Brick Wall (2006):

Power Wall + Memory Wall + ILP Wall = Brick Wall

"infinite" IO/Sec and virtual-RAM with PCI-SSD. eg Fusion-IO

With cheap PCI-SSD by the Terabyte, the majority of Apps/enterprises don't need:

Big Iron Databases
Big Iron Storage Arrays and supporting SAN's
Big Iron multi-chip fast uniprocessing cores.

A lot of the complexity of Big Iron DB's (like Oracle) is aimed at achieving "speed" in the face of low-performing HDD's... [slow IO/sec, not streaming throughput]

If the whole of a relational DB (tables) fits in memory (or fast Virtual memory), then doesn't the DB become very simple, modulo ACID tests and writing "commits" to persistent, high-reliability storage?

Which means we might start seeing a bunch of in-memory DB's, like NoSQL, but for normal-sized DB's (1-5Gb), not large collections.

There's an economic rule on product substitution that led to the relatively quick decline in IBM mainframe sales:

when the capital expenditure on a substitute is less than the operational costs of the current system, barriers to adoption are removed.

capital costs are 'sunk' and can't be recovered.
To realise savings, you have to wait for the next upgrade or refresh cycle.
But the conversion costs have to be factored in, and incumbent vendors take care to price upgrades, even "forklift upgrades" (complete replacement) under the total cost of moving to a new solution. [Used to advantage by Tier 1 Storage Array vendors currently].
But Operational Costs, like maintenance charges, aren't 'sunk'.

They are due every year.
When a whole system is less than recurrent costs, businesses can quickly and easily justify the change.
They write-off the CapEx for the old equipment and wheel it out the door.
Usually "Big Iron" hardware has zero residual value, even when 2 years old.
When the equipment is on the cusp of being obsolescent, it is worse.
Or they have to figure out how to break the lease.
Unless they went into debt to fund the CapEx, they can move away quickly.

Hardware maintenance fees are typically 15-20% of capital costs.
Whilst Oracle licensing costs are beyond me (I don't track them) - but are becoming a major component of Enterprise Computing costs.

How many "little" DB applications need to succeed with two low-cost ($10k) servers, 3 SATA drives each in simple RAID and 1 Fusion-IO board, run as H/A with an in-memory DB?

If organisations can build a complete, high-performance, high-availability, simple-admin solution for $20-$30k per group of DB's, they can afford to deploy them immediately based on direct maintenance savings.

Follow-up comment:

Intel are looking over their shoulder at becoming dinosaurs. Maybe I won't live to see it, but ARM servers could very well do in the x86_64.
Ed

And so they should for most general purpose computing.

I remember seeing John Mashey of MIPS talk in 1988 where he plotted CPU speed for each of ECL, bipolar and CMOS technologies. ECL had been overtaken by then, bipolar was due to lose the lead within a few years.
The 486 in 1991 was a complete system-on-a-chip and changed the landscape.

It answers the question "Where did all the supercomputers go?"
A: Inside Intel. [and Power and SPARC. possibly Z series]

The Intel chips seek "maximum performance" - they pull all the tricks that super-computer designs used, and its is that technology that is approaching Pattersons' "Brick Wall" [heat, memory, ILP]

And as an aside, GPU's are filling the "vector processor" niche of CDC and Cray.

ARM has pursued a very different strategy, more based around 'efficiency': MIPS/Watt

So, while I agree with you, I think the situation is nuanced.

ARM processors are obvious choices for low-power and mobile/battery devices.
Because of design simplicity (small PSU, no CPU-fan) and smaller size, they'll become more interesting for low-end PC's, especially portable devices.

There is a company, Calxeda, now producing high-density ARM boards for servers.
They are hoping to leverage MIPS/Watt for highly-parallelisable loads, like web-servers.

But I can't see anyone taking on Intel soon in the supercomputer-on-a-chip market.
It's not just servers, especially for large DB's, but workstations and 'performance' laptops.

The problem with that evolution of the market for Intel is ARM taking sales from multiple market segments. Seeing that Winders-8 will run on ARM, we might see the end of WinTel for low-end & mid-tier laptops.

As a company, can Intel survive such a radical change in demand for its major product line?
Will its work on MLC flash fill the financial void?

I've no idea how that will go.
But like you said, ARM is going to shake up even the Intel server market.

The "secret sauce" that the ARM architecture has is that it's a licensed design.
Although chip design companies might not own or be able to access chip FABs within 2 or 3 design cycles of Intel, they can produce highly optimised and use-case targeted chips.

Which Intel can't do. They are focussed on the bleeding edge of CPU performance and FAB design.

Manufacturers like Apple/A5 and Calxeda can produced ARM-based designs that can outperform Intel-based systems by an order-of-magnitude on non-MIPs metrics.

As Apple has shown, there are very big markets where raw MIPs isn't the "figure of merit" in designs.

2011/11/30

Apple needs to invent 'The Brick': screenless high-performance CPU, Graphics and Storage

Two new things appeared in my world recently:

A friend showed me Google 'Jump Desktop', connecting his iPad to his Desktop. [Apple have their own 'Remote Desktop' product which I haven't researched. Not sure if it works over the wider Internet as well.]
Joe Wilcox's piece (Jan-2011) on Apple dumping desktops sometime (soon) as they become less relevant.

Wilcox wonders what will happen to "Power Users" if Apple move on Desktops, as they've moved on from the rack-mount X-serve line. Customers needing servers now have the choice of a Mac Mini Server or a "Mac Pro" (tower).

Can Apple afford to cut-adrift and ignore the needs of good folk who rely on their product for their business/livelihood, some of whom may have used Macs for 20+ years?? Would seem a Bad Idea to alienate such core and influential users.

Clearly Apple look to the future, and like the floppy drive they expunged long ago in favour of Optical drives (now also obsolete), Desktops as we know them are disappearing from mainstream appeal and usefulness.

I think there are two markets that Apple needs to consider:

One they haven't won yet: Corporate Desktops, and
One that's been part of their core business for decades: High-end Graphics/Media

Thunderbolt on laptops means big, even dual, monitors are simple for Corporate Desktops, addressing a large part of the demand/needs. While Apple retain the Mac Mini line, they have a viable PC Desktop replacement for those organisations that like the "modular PC" model, especially those that don't want laptops walking out the door.

The simplicity and elegance of Just One Plug of the iMac makes it unbeatable in certain niche applications, such as public use PC's in Libraries or battery workstations in call centres.

Can Apple produce a "power" laptop with the processing, graphics and storage size/performance that meets the needs of High-end Media folk?
A: No, never. Because the fastest, most-powerful CPU's, GPU's, most RAM and largest/fastest storage only ever come with high-power and big footprint: you need a big box with a big power supply: The definition of a Desktop or Workstation.

One solution would be to licence OS/X to "tier 1" PC vendors like Dell or HP for use on certified systems. But that's not going to happen, Apple is a hardware/manufacturing company - they will never go there.

Hence "the Brick" that is mainly accessed via "Remote Desktop".
My suggestions are a modular design, not dissimilar to the NGEN's expandable 'slices':

CPU's and RAM in a housing with capacity to gang together for scale-up.
GPU's in a PCI-slot chassis, with Thunderbolt available for physical displays.
Local storage via e-SATA, SAS or Thunderbolt.
remote bulk storage over the network
External power-supply, or part of a base-unit (CPU, RAM, PCI-slot, network, Thunderbolt).

The point of "the Brick" is ComputePower-on-Demand and Universal-Workspace-View, not unlike SUN's 1993 "Starfire video" prototype.
It can live in a (locked) cupboard, or many can be hosted on a server cluster as one of many Virtual Machines. For even a modest operation, high-power servers running VMware makes operational and economic sense. VM's mean another licensing deal. Perhaps VMware, part of EMC, might have the clout to do a deal like this with Apple. Or not.

Jim Gray authored a paper in 2004, "TerraServer Bricks" as an alternative architecture. The concept is not new/original and more than the usual low-power appliances.

An aside on "Jump Desktop", it uses well established (and secure) remote desktop protocols (RDP, VNC). But for Unix/Linux users interested in security and control, this is important:

Jump also supports SSH tunneling for RDP and VNC connections which also adds a layer of encryption but this must be configured manually.

2011/09/14

A new inflection point? Definitive Commodity Server Organisation/Design Rules

Summary:

For the delivery of general purpose and wide-scale Compute/Internet Services there now seems to be a definitive hardware organisation for servers, typified by the E-bay "pod" contract.

For decades there have been well documented "Design Rules" for producing Silicon devices using specific technologies/fabrication techniques. This is an attempt to capture some rules for current server farms. [Update 06-Nov-11: "Design Rules" are important: Patterson in a Sept. 1995 Scientific American article notes that the adoption of a quantitative design approach in the 1980's led to an improvement in microprocessor speedup from 35%pa to 55%pa. After a decade, processors were 3 times faster than forecast.]

Commodity Servers have exactly three possible CPU configurations, based on "scale-up" factors:

single CPU, with no coupling/coherency between App instances. e.g. pure static web-server.
dual CPU, with moderate coupling/coherency. e.g. web-servers with dynamic content from local databases. [LAMP-style].
multi-CPU, with high coupling/coherency. e.g. "Enterprise" databases with complex queries.

If you're not running your Applications and Databases in Virtual Machines, why not?
[Update 06-Nov-11: Because Oracle insists some feature sets must run on raw hardware. Sometimes vendors won't support your (preferred) VM solution.]

VM products are close to free and offer incontestable Admin and Management advantages, like 'teleportation' or live-migration of running instances and local storage.

There is a special non-VM case: cloned physical servers. This is how I'd run a mid-sized or large web-farm.
This requires careful design, a substantial toolset, competent Admins and a resilient Network design. Layer 4-7 switches are mandatory in this environment.

There are 3 system components of interest:

The base Platform: CPU, RAM, motherboard, interfaces, etc
Local high-speed persistent storage. i.e. SSD's in a RAID configuration.
Large-scale common storage. Network attached storage with filesystem, not block-level, access.

Note that complex, expensive SAN's and their associated disk-arrays are no longer economic. Any speed advantage is dissolved by locally attached SSD's, leaving only complexity, resilience/recovery issues and price.
Consequentially, "Fibre Channel over Ethernet" with its inherent contradictions and problems, is unnecessary.

Designing individual service configurations can be broken down into steps:

select the appropriate CPU config per service component
specify the size/performance of local SSD per CPU-type.
architect the supporting network(s)
specify common network storage elements and rate of storage consumption/growth.

Capacity Planning and Performance Analysis is mandatory in this world.

As a professional, you're looking to provide "bang-for-buck" for someone else who's writing the cheques. Over-dimensioning is as much a 'sin' as running out of capacity. Nobody ever got fired for spending just enough, hence maximising profits.

Getting it right as often as possible is the central professional engineering problem.
Followed by, limiting the impact of Faults, Failures and Errors - including under-capacity.

The quintessential advantage to professionals in developing standard, reproducible designs is the flexibility to respond to unanticipated load/demands and the speed with which new equipment can be brought on-line, and the converse, retired and removed.

Security architectures and choice of O/S + Cloud management software is outside the scope of this piece.

There are many multi-processing architectures, each best suited to particular workloads.
They are outside the scope of this piece, but locally attached GPU's are about to become standard options.
Most servers will acquire what were known as vector processors and applications using this capacity will start to become common. This trend may need their own Design Rule(s).

Different, though potentially similar design rules apply for small to mid-size Beowulf clusters, depending on their workload and cost constraints.
Large-scale or high-performance compute clusters or storage farms, such as the IBM 120 Petabyte system, need careful design by experienced specialists. With any technology, "pushing the envelope" requires special attention by the best people you have, to even have a chance of success.

Not unsurprisingly, this organisation looks a lot like the current fad, "Cloud Computing" and the last fad, "Services Oriented Architecture".

Google and Amazon dominated their industry segments partly because they figured out the technical side of their business early on. They understood how to design and deploy datacentres suitable for their workload, how to manage Performance and balance Capacity and Cost.

Their "workloads", and hence server designs, are very different:

Google serves pure web-pages, with almost no coupling/communication between servers.
Amazon has front-end web-servers is backed by complex database systems.

Dell is now selling a range of "Cloud Servers" purportedly based on the systems they supply to large Internet companies.

2010/08/29

Datacentre Hardware organisation

Senior Google staffers wrote The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, which I thought showed break-through thinking.

The importance of this piece is it wasn't theoretical, but a report of What Works in practice, particularly at 'scale'.

Anything that Google, one of the star performers of the Internet Revolution, does differently is worthy of close examination. What do they know that the rest of us don't get?

While the book is an extra-ordinary blueprint, I couldn't but help asking a few questions:

Why do they stick with generic-design 1RU servers when they buy enough for custom designs?
How could 19-inch racks, designed for mechanical telephone exchanges a century ago, still be a good, let alone best, packaging choice when you build Wharehouse sized datacentres?
Telecommunications sites use DC power and batteries. Why take AC, convert to DC, back to AC, distribute AC to every server with inefficient, over-dimensioned power-supplies?

Part of the management problem with datacentres is minimising input costs whilst maximising 'performance' (throughput and latency).

Everything Old is New Again: Cray's CPU design

I found myself writing, during a commentary on the evolution of SSD's in servers, that large-slow-memory like Seymour Cray used (not cache), would affect the design of Operating Systems. The new scheduling paradigm:

Allocate a thread to a core, let it run until it finishes and waits for (network) input, or it needs to read/write to the network.

This leads into how Seymour Cray dealt with Multi-Processing, he used multi-level CPU's:

There were Application processors, many bits, many complex features like Floating Point and other fancy stuff, but had no kernel mode features or access to protected regions of hardware or memory, and
Peripheral Processors (PP's), really a single very simple, very high-speed processor, multiplexed to look like 10 small, slower processors that performed all kernel functions and controlled the operation of the Application Processors (AP's)

Not only did this organisation result in very fast systems (Cray's designs were the fastest in the world for around 2 decades), but very robust and secure ones as well: the NSA and other TLA's used them extensively.

The common received wisdom is that interrupt-handling is the definitive way to interface unpredictable hardware events with the O/S and rest of the system. That polling devices, the old-way, is inefficient and expensive.

Creating a fixed overhead scheme is more expensive in compute cycles than an on-demand, or queuing, system, until the utilisation rate is very high. Then the cost of all the flexibility (or Variety in W. Ross Ashby's Cybernetics term) comes home to roost.

Piers Lauder of Sydney University and Bell Labs improved total system throughput of a VAX-11/780 running Unix V8 under continuous full (student/teaching) load by 30% by changing the serial-line device driver from 'interrupt handling' to polling.

All those expensive context-switches went away, to be replaced by a predictable, fixed overhead.
Yes, when the system was idle or low-load, it spent a little more time polling, but marginal.
And if the system isn't flat-out, what's the meaning of an efficiency metric?

Dr Neil J Gunther has written about this effect extensively with his Universal Scaling Law and other articles showing the equivalence of the seemingly disparate approaches of Vector Processing and SMP systems in the limit of their performance.

My comment about big, slow memory changing Operating System scheduling can be combined with the Cray PP/AP organisation.

In the modern world of CMOS, micro-electronics and multi-core chips, we are still facing the same Engineering problem Seymour Cray was attempting to address/find an optimal solution to:

For a given technology, how do you balance maximum performance with the Power/Heat Wall?

More power gives you more speed, this creates more Heat, which results in self-destruction, the "Halt and Catch Fire" problem. Silicon junctions/transistors are subject to thermal run-away, as they get hotter, they consume more power and get hotter still. At some point that becomes a viscous cycle (positive feedback loop) and its game over. Good chip/system designs balance on just the right side of this knife edge.

How could the Cray PP/AP organisation be applied to current multi-core chip designs?

Separate the CPU designs for kernel-mode and Application Processors.
A single chip needs only have a single kernel-mode CPU controlling a number of Application CPU's. With its constant overhead cost already "paid for", scaling of Application performance is going to be very close to linear right up until the limit.
Application CPU's don't have forced context switches. They roar along as fast as they can for as long as they can, or the kernel scheduler decides they've had their fair share.
System Performance and Security both improve by using different instruction sets and processor architectures for different applications. While a virus/malware might be able to compromise an Application, it can't migrate into the kernel unless it's buggy. The Security Boundary and Partitioning Model is very strong.
There doesn't have to be competition between the kernel-mode CPU and the AP's for cache memory 'lines'. In fact, the same memory cell designs/organisations used for L1/L2 cache can be provided as small (1-2MB) amounts of very fast direct access memory. The modern equivalent of "all register" memory.
Because the kernel-mode CPU and AP's don't contend for cache lines, each will benefit hugely in raw performance.
Another, more subtle, benefit is the kernel can avoid both the 'snoopy cache' (shared between all CPU's) and VM systems. It means a much simpler, much faster and smaller (= cooler) design.
The instruction set for the kernel-mode CPU will be optimised for speed, simplicity and minimal transistor count. You can forget about speculative execution and other really heavy-weight solutions necessary in the AP world.
The AP instruction set must be fixed and well-know, while the kernel-mode CPU instruction set can be tweaked or entirely changed for each hardware/fabrication iteration. The kernel-mode CPU runs what we'd now call either a hypervisor or a micro-kernel. Very small, very fast and with just enough capability. A side effect is that the chip manufacturers can do what they do best - fiddle with the internals - and provide a standard hypervisor for other O/S vendors to build upon.

Cheaper, Faster, Cooler, more robust and Secure and able to scale better.

What's not to like in this organisation?

SteveJ-on-IT