In the coming weeks, the feds and the surviving financial services institutions will have the daunting task of unraveling all the securitized loans and other instruments that are hiding the toxic investments. But does the technology exist to do that? And if so, could it have been used to prevent the bad debt from hitting the fan in the first place?

The fact is that despite government regulations like Sarbanes-Oxley, there is little visibility mandated by current regulations into the origination of loans and how they are broken up, resold, and resold again.

[ After-the-fact BI doesn't help identify problems early. Read how operational analytics could help flag issues so that you can act before they blow up. ]

To cite the classic example of how we got into this mess, consumers were given 100-percent-plus variable mortgages without any security. Not only could those mortgages be sold to other banks, but they could be divided into 5, 10, or 20 tranches -- financialese for slices -- and resold to between 5 and 10 different organizations, making it difficult to track who was involved and who ended up taking the risk.

Theoretically, the financial service providers were clear on the risks of each type of loan and had a way to gauge whether they had enough liquidity -- cash and other easily sold assets -- available if the riskier loans went south. But a New York Times report indicates that in fact many financial institutions gamed their analytics to favor positive scenarios over negative ones in order to justify keeping less money in reserves should the risky loan blow up. "A large number of buyers of these kinds of instruments really didn't care about the value. They just wanted to flip it. A lot of people just didn't want to know," says Josh Greenbaum, principal at Enterprise Applications Consulting.

Analytics and CEP tools could have helped Had these financial services companies and banks established business intelligence metrics as to the ratios of what kind of debt they were holding versus the cash reserves they held, their analytics systems might have driven alerts earlier in the process, says Michael Corcoran, a product manager at the BI provider Information Builders. But as anyone in business already knows, consolidating that kind of data to get those answers more often than not is a slow process that typically ends up being done manually in an Excel spreadsheet well after the fact.

Jeff Wooton, vice president of product strategy at Aleri, a CEP (complex event processing) company, agrees that most data consolidation takes far too long to give a complete picture. "It relies on overnight data consolidation runs, overnight reports, and manual processes like spreadsheets."

That's where technologies such as CEP and operational BI come into play. They analyze huge volumes of transactions -- 100,000 messages per second with millisecond response time, triggering remedial actions by other systems. But they can also be slowed down and used by analysts as a decision support tool. Tools such as Aleri's Liquidity Management System already exist to help treasurers in global banks gauge their liquidity position in real time. Wooton says that over the last two weeks there has been considerably more interest in such products than in the past.

Wooton cautions that the various kinds of analytics tools available, such as business activity monitoring, decision support software, data integration, and alerts, could have offered a warning but not fixed the underlying problem of financial services firms misjudging -- and in some cases, misrepresenting -- the risks of their loans and securities.

But taking analytics, CEP, and data integration to the next level to give regulators a sense of what all the financial institutions were doing and what the liquidity risks actually were could have helped, and it could prevent a recurrence, Corcoran says. He says that the use of middleware could bring the data together and create a "common front end" that is shared by regulators and the services companies alike.

But that front end needs a common back end, especially around the data that should exist, says consultant Greenbaum. "The data model for doing the analysis doesn't exist," he says, so a company selling securities and packaged mortgages doesn't include the packaging history. "You can't do the classic drill-down," Greenbaum says, because no one knows what the relationships are because the metadata hasn't been preserved -- or at least not preserved in a way that is easy to find.

Integration is a double-edged sword While integration of data and the analytics around financial investments could help prevent future financial meltdowns, it's also true that the integration of business activities on a global scale helped get us into this crisis in the first place, says Suzanne Duncan, financial markets industry leader for the Institute for Business Value at IBM.

"Firm-to-firm and country-to-country integration is increasing. This improves efficiency because it lets capital flow to where it is needed, but at the same time these linkages cause larger shocks at a greater frequency," Duncan says.

The reason comes back to the lack of visibility into financial risks and liquidity both at the individual financial institution and globally. Because the information is too scattered throughout the firm and held in silos of information systems, many financial institutions did not even have a grasp on the loans they held, he says.

"Many of these bigger firms don't know what their counterparty exposures are. They don't know how much Lehman Brothers owes them," Wooton adds. (A counterparty is any organization with which your company has some kind of relationship with, be it as partner or as a client.) Duncan agrees: "No matter which part of the ecosystem you are talking about, companies don't know what their counterparty exposures are." IBM is in the midst of working with its Asian clients to root through what their exposures are.?

Retail algorithms may be reused for financial services Although much of the technology already exists that could have tracked and helped to at least warn companies of the dangers ahead, IBM is also looking at retooling some current technology that uses sophisticated algorithms to map processes to help increase the visibility into risks of financial instruments that are dispersed globally.

Up until now, technology in the financial services industry has been focused on capacity -- whether an application can handle high volume and volatility -- rather than on process. There is no process flow map that tells organizations who owns what pieces of what risk.

But in the retail industry, there are such flow-mapping technologies to track, for example, that consumer A buys a car and two years later sells that car to consumer B, who in turn sells his car to consumer C, while consumer A buys a new car; the software maps all of those processes for the sake of tracking buyer behavior. By scanning through the Web and physical public documents, a retailer puts together a point of view on a customer's buying behavior. "That kind of process mapping hasn't been unleashed to track the whole world of institutional behavior," Duncan says. But perhaps it could.

Of course, even with the right technology in place, Greenbaum cautions, it will do no good unless meaningful government oversight is put in place, so the financial services companies can't again -- through ignorance or deceit -- so wildly create and distribute toxic assets.

Ruby on Rails 2.2, an upgrade to the popular Web application framework, was released Friday, featuring an internationalization framework and stronger support for HTTP validators, according to the Ruby on Rails Web site.

With a full-on internationalization framework, internationalization is offered by default. Support for HTTP validators is provided in the form of etag and last-modified, according to the site. This can make it easier to skip expensive processing and also makes it easier to use gateway proxies.

Also featured are thread safety and a connection pool for the Active Record capability in Rails. "So now all elements of Rails are thread-safe, which is a big boon for the JRuby guys in particular," a blog on the site stated. "For C Ruby, we still need a bunch of dependent libraries to go non- blocking before it'll make much of a difference, but work on that is forthcoming."

Rails 2.2 also features improved API docs and a new guides section. It is compatible with Ruby 1.9 and JRuby.

Connection pooling in version 2.2 enables Rails to distribute requests across a pool of databases, according to release notes for the framework. Transactional migrations in version 2.2 are supported on PostgreSQL out of the box. The code will be extensible to other database types in the future, the notes said.

The framework can be installed through the RubyGems packaging system for Ruby.

Ruby on Rails was created by David Heinemeier Hansson, and Ruby and Ruby on Rails were featured at this week's QCon conference in San Francisco.

Yahoo CEO Jerry Yang, who co-founded the company, stepped aside this week to the surprise of no one who has followed the recent travails of the company. While that ship continued to list, the IETF debated what, if anything, to do about the problematic DNS bug that was discovered earlier this year. And the BlackBerry Storm lived up to its name, if not entirely to its hype, as it debuted in the United States and the United Kingdom.

[ Video: Catch up on the week's news with the World Tech Update ]

1. Yahoo's Yang to step down as CEO and What's Yahoo's next move?: From the "it's about time" file -- Yahoo co-founder Jerry Yang is leaving the company's CEO post, which he took over in July 2007. While he took charge to try to right what was wrong with the company, his CEO tenure didn't go so well, with the failed Microsoft buyout attempt, followed by a failed ad deal with Google, with two rounds of layoffs and slumping finances mixed in. Yang will continue as a board member, and when a new CEO is found he will resume his previous title of "Chief Yahoo." What happens next at Yahoo has been the source of much debate this week.

2. A future without programming: There are presently tons of codeless app dev tools available, tools that will help you create an app without having to do any of the coding yourself. This is a great boon to people who want to create simple apps without having to write all the code, or even noncoders who just want to make the app that they need themselves. But they could also be signaling a decline for developers as they see themselves replaced by applets.

3. Microsoft, Novell eye Moonlight beta, system management: As Microsoft and Novell near the two-year anniversary of their controversial interoperability agreement, they are announcing a beta of Moonlight, which will bring Microsoft's Silverlight RIA technology to Linux. The companies are also rolling out the Advanced Management Pack, which enables management of Windows and Linux servers from a single console.

[ For a two-year retrospective on the agreement, featuring comments from Microsoft, Novell and an opponent of the arrangement, see The Microsoft-Novell Linux deal: Two years later. ].

4.JavaFX RIA technology almost ready: Sun says that general release products for JavaFX Desktop and JavaFX Script should be out by the end of the year. Featuring an application platform based on Java, a scripting component and runtimes for desktop and mobile systems, JavaFX, Sun officials said, gives the company a unique entrant in a market also featuring Adobe Systems, with Flash, and Microsoft, with Silverlight.

5. Five top spending priorities for hard times: Forrester, Gartner and IDC have slashed 2009 IT spending growth projections, with IDC forecasting a decline to a paltry 0.9 percent from its pre-financial crisis prediction of 4.2 percent. Analysts say that despite the grim financial scene, companies should not inflict deep cuts on IT. "Companies should tighten their belts, not take their pants off," says Forrester senior analyst Andrew Reichman. InfoWorld chatted up analysts and CIOs to find out which technologies should be funded regardless of what the economy is up to (or down to, as it were).

6. Hosted Exchange, SharePoint now widely on sale: About 500,000 users have already adopted Exchange Online since a limited release for large enterprises in October 2007, and Microsoft expects half of all enterprise employees with e-mail to use a combined online and premises-based system in five years. Naturally, the company hopes that its Exchange Online, now in full release, will capture a large part of that market.

7. Survey: U.S. IT spending forecast worst since 2001: Forty-five percent of those who responded to a new ChangeWave Research Survey said their companies aim to spend less on IT or even nothing at all on IT during the first quarter of 2009 -- the highest percentage response to that survey question since 2001. Researchers talked to 1,926 U.S. respondents who are involved with IT spending to get the dismal survey results. Just 10 percent said they plan to spend more in the first quarter, which was down three points from an August survey.

8. Microsoft drops OneCare anti-virus product: Microsoft is essentially giving up on its efforts to build a consumer anti-virus business as it is discontinuing its OneCare software. Microsoft pushed hard to get OneCare to be thought of as on the same level as products from Symantec or McAfee, but it was poorly reviewed and failed to establish itself as being able to run with the big boys. OneCare will be replaced with free anti-virus software called Morro.

9. Obama administration to inherit tough cybersecurity challenges: The administration of President-elect Barack Obama will wind up dealing with key cybersecurity initiatives begun during the Bush administration, but far from fruition. More progress has been made on other cybersecurity projects, but some of those have been found to be lacking. Those in the security industry say that the next administration will also have to focus on collaboration between public and private sectors.

10. Bush's exit to put new e-records system to the test: The National Archives and Record Administration expects to receive 140TB of data when the Bush administration ends after eight years with inauguration day, Jan. 20, when all paper and electronic records of the administration become the legal responsibility of NARA. The unprecedented volume of records has to be sorted, indexed, and preserved. Comparatively, the Bush years are expected to generate 50 times more data than the Clinton administration, which also went two terms.

Nemertes Research continued to throw cold water on the future of the Internet this week, releasing a study projecting that demand for bandwidth on the Web would exceed its capacity by 2012.

The study, which is a follow-up to similar research Nemertes conducted last year, projects that the current global economic recession will only delay rather than eliminate the increased demand for bandwidth the firm predicted last year. Then, Nemertes projected that traffic growth would eclipse supply by 2010, but the firm now says it has adjusted its projections to reflect deteriorating global economic conditions.

[ Does the bandwidth shortage mean out Internet future is in danger? ]

Nemertes emphasized it is not projecting that the Internet will crash or shut down altogether. Rather, the typical user probably will experience Internet "brownouts," where such high-bandwidth applications as high-definition video-streaming and peer-to-peer file-sharing will stop performing up to users' expectations, the firm says. 

During a presentation at an Internet Innovation Alliance symposium this week, Nemertes analyst Mike Jude said that one consequence of declining Web performance would be that users would look less to the Internet to deliver their desired applications. "More and more applications are coming online that will drive expectations for service quality even higher," he said. "I'm not saying that the Internet is going to crash in 2011, but that people's expectations are going to be throttled. People will stop going to the Internet for those services."

One big reason for the projected growth in traffic is the continuing emergence of virtual workers who work from home or in remote branch offices located far away from companies' central offices, Nemertes says. In particular, these remote workers "expect seamless communications, regardless of where they conduct business" and they "often require more advanced communication and collaboration tools than those who work at headquarters," including videoconferencing and Web conferencing, the report says.

Another factor is simply the large growth in high-bandwidth applications for users to employ. More ISPs in the coming years will follow the lead of such companies as Comcast and AT&T trying out bandwidth caps that will charge extra money each month for heavy bandwidth consumers, Nemertes says. Although Comcast now caps individual bandwidth consumption at a relatively high 250GB per month, average future users will easily reach or surpass that bandwidth limit as they find higher-bandwidth applications to use, the firm says.

"Though this traffic load is [currently] more than typical, it certainly isn't exceptional," Nemertes reports. "This type of usage will become typical over the next three to five years. The fact that Comcast's network is, by its own admission, not able to cope with such usage patterns is a clear indication that the crunch we predicted last year is beginning to occur."

Looking forward, Nemertes says that if this capacity issue is not addressed, the Internet will fracture into a tiered system where companies with the most money will pay for specialized network infrastructure that will ensure their content is delivered at higher speeds than non-favored content.

This fractured system -- where certain entities can pay extra money to give their content favored treatment -- is what advocates of network neutrality have been working to avoid by preventing ISPs from discriminating against certain types of content. The Nemertes report gloomily concludes that although the Internet will not shut down entirely, it will experience a dramatic slowdown in innovation because "new content and application providers will be handicapped by the relatively poorer performance of their offerings vis-à-vis those created by the established players."

Microsoft asked a federal judge Thursday to end the class-action lawsuit about its "Vista capable" tag that has been the source of a treasure trove of embarrassing insider e-mails that have showed the company bent to pressure from Intel and infuriated longtime partner Hewlett-Packard.

In a pair of motions filed with U.S. District Court Judge Marsha Pechman, Microsoft's lawyers asked her to decertify the class and rule on a summary judgment to dismiss the charges.

[ InfoWorld's Robert X. Cringely did a rundown of the legalese of this case -- and the humor of it ]

If Pechman rules for Microsoft on the decertification motion, the case could conceivably continue, although it would no longer be a class-action with a large pool of plaintiffs; instead, each plaintiff would have to sue Microsoft separately. A ruling for the company on the summary judgment would effectively end the case.

Unlike recent filings by the plaintiffs, which have been packed with quotations from internal Microsoft e-mails that covered everything from managers badmouthing Intel to others who worried how Vista would be compared to Apple's Mac OS X, Microsoft's motions were densely worded and full of case citations.

According to Microsoft, the plaintiffs have not demonstrated that the lowest-priced version of Windows Vista was not the "real" Vista, or showed that users paid more for PCs prior to the new operating system's launch because of the Vista Capable campaign. That means the plaintiffs have not met the legal standards set by Pechman, and so have no case, the attorneys argued.

"The evidence refutes Plaintiffs' claims that Windows Vista Home Basic cannot 'fairly' be called Windows Vista," Microsoft said in the motion for summary judgment. "Windows Vista Home Basic has nearly all of the same computer code as the rest of the Windows Vista family, and ... Microsoft never publicly defined Windows Vista in a way that would exclude Windows Vista Home Basic."

Vista Home Basic, the lowest-priced and least-capable version of the operating system, is a key to the Vista Capable lawsuit; the plaintiffs have argued that they bought PCs before Vista's January 2007 launch and expected them to be able to run more than just Home Basic. That edition lacks several advanced features found in some or all of the other versions, notably the Aero graphical user interface.

Elsewhere in the motion, Microsoft claimed that Vista Home Basic shared 93 percent of the code found in Vista Home Premium, the next-most-expensive version and also the most popular of the consumer editions.

The lawyers also hammered at the price inflation reasoning promoted by the plaintiffs. "Plaintiffs have no evidence that the Windows Vista Capable program ('WVC program') caused an artificial increase in the demand for or prices of Windows Vista Capable PCs ('WVC PCs') that were not Premium Ready," the motion continued.

Last February, when Pechman granted the case class-action status, she blocked the plaintiffs from arguing that Microsoft deceived consumers because that would have required an individual determination for each member of the class action. Instead, she allowed them to pursue a "price inflation" line of reasoning, which would argue that PC buyers paid more than they would have otherwise, after Microsoft's marketing boosted demand and increased the prices of systems that could run Vista Home Basic.

In the motion to decertify the class, Microsoft's lawyers said that the plaintiffs had not met the bar Pechman set when she allowed them to explore the price inflation line. "With discovery closed, Microsoft asks the Court to decertify the class because Plaintiffs have done nothing and propose to do nothing to further develop their price inflation theory," Microsoft said.

Discovery, the legal procedure where the each party is allowed to request documents from other, closed a week ago in the case. "The Plaintiffs have no viable method of establishing class-wide causation," the motion continued.

Over several pages, Microsoft argued that the economist the plaintiffs brought in as an expert witness, Keith Leffler, of the University of Washington, had been unable to come up with a way to quantify the impact of the Vista Capable program on PC prices in the run-up to Vista.

"Dr. Leffler admitted that he cannot develop a model that would quantify the price inflation, if any, that supposedly affected the class, much less do so across the entire class period," the motion said. "That fact, standing alone, mandates decertification."

Microsoft crafted the Vista Capable program to keep sales of PCs from flagging as the new OS's release loomed. A message by a Microsoft director working on the campaign made it clear that was the top priority. "The primary goal of Ready PC [ an earlier name for what would be recast as Vista Capable -- Ed. ] is to limit stall of XP PC sales as we continue to build Vista buzz," said Rajesh Srinivasan in October 2005. "We believe [the program requirements] strike a balance between limiting impact on XP PC sales, ensuring OEM support and participation in the program and providing a good customer experience after Vista upgrade."

The lawsuit, which began in April 2007, has become best-known as the source for hundreds of Microsoft e-mails that have been made public by the court. Earlier disclosures showed that Microsoft relaxed the requirements of Vista Capable to accommodate Intel, a decision that then enraged HP, and that company managers feared comparisons between Vista and Apple's Mac OS X more than a year before Vista went public.

The case is currently set to start trial next April.

Computerworld is an InfoWorld affiliate.

In separate moves this week, Sun and Microsoft both proceeded with previously stated plans to boost their software development environments

Version 6.5 of the NetBeans open source IDE was released by Sun and the NetBeans community, while Microsoft has added jQuery IntelliSense support to Visual Studio 2008 and Visual Web Developer 2008 Express.

Accessible for download, NetBeans 6.5 features increased support for Web and Java software development, according to Sun and the NetBeans community. It includes localized versions for simplified Chinese, Japanese, and Brazilian Portuguese.

Also being offered is an early access version of NetBeans for Pythin applications, featuring an editor, debugger, and Python runtimes.

Version 6.5 features tooling for PHP, such as syntax highlighting and code completion. A JavaScript editor is included as well.

?Integration across multiple languages simplifies development. The NetBeans IDE 6.5 allows you to stay within one tool and move easily from PHP to JavaScript and back," said Ian Murdock, Sun vice president of developer and community marketing at Sun, in a statement released by the company.

Other capabilities include enhanced support for Spring, Hibernate, JavaServer Pages, and Java Persistence API. Support for Groovy and Grails also is offered in the editor. Ruby enhancements are offered within the editor and debugger.

Multithreaded debugging for Java technologies is featured as well.

Sun in December will offer a training and certification for NetBeans by way of its Certification Specialist for NetBeans IDE effort.

Microsoft, meanwhile, is offering JavaScript IntelliSense support via Service Pack 1, which can be downloaded. JQuery is a JavaScript library.

Users also must install the VS 2008 Patch KB58502 patch to support "-vsdoc.js" Intellisense files and download the jQuery-vsdoc.js file.

"Visual Studio 2008 SP1 adds richer JavaScript IntelliSense support to Visual Studio, and adds code completion support for a broad range of JavaScript libraries," said Scott Guthrie, corporate vice president in the Microsoft Developer Division, in his blog.