Picking Software Programs: Know Their Limitations

By: Louis B. Mendelsohn
President, Market Technologies Corporation

The microcomputer no longer is a novelty in futures markets. Neither are the software packages that now are readily available in many forms, some promising to be that “holy grail” which commodity traders continue to seek.

But at this point, with many traders convinced a microcomputer is THE solution to their trading needs, it is time to take a look at what the present generation of software programs can or cannot do for you and to suggest guidelines for future software development.

As technical trading software has become more sophisticated and widely used, minimum hardware standards have advanced rapidly. Cassette tape recorders have been replaced by floppy disk drives as storage media, television sets have been replaced by video monitors and micro-modems have become necessary for telecommunications.

Today’s minimum
The lowest common denominator to which most commodity trading software now is tailored requires a 48K computer, two floppy disk drives, a video monitor, printer and micro-modem. This system configuration allows each trader to run a wide range of commodity software programs without necessitating costly software customization or hardware upgrading. Software vendors also benefit by being able to market their software to a large, relatively homogeneous population of traders.

As the application of computer technology to commodity trading plays an increasing role in the decision-making process, of course, system standards for both hardware and software will continue to be redefined and upgraded.

Commodity trading software can be classified into two broadly defined categories:

Analysis or “smorgasbord” type.
System or “black box” software.

Before responding to any software pitch, you should know the characteristics and limitations of each.

Analysis software
Analysis software allows you to choose from a “smorgasbord” of user selectable “entrees.” By combining some of these, you can put together or customize your own trading plan. Studies that can be performed include different types of moving averages, relative strength indicators, oscillators and momentum.

This approach has a broad appeal to commodity traders because of the large assortment of entrees available and the ability to develop customized trading plans. Because this software does not produce bunched signals, a vendor does not have to be concerned about causing bad executions and does not have to restrict sales.

Due to the large assortment of entrees that can be selected, this type of software usually includes an autorun capability to help reduce some of the clerical activities. The vendor promoting this type of software also usually offers ongoing user support in the form of periodic updates and enhancements for a fixed annual fee.

However, the smorgasbord concept also is directly responsible for some of the most serious limitations of this type of software. These shortcomings cannot be remedied easily within this type of software design:

This software is primarily geared toward number-crunching. You still must devise your own trading plan by using one or more entrees, and you still must personally analyze the crunched numbers to arrive at a trading signal.
You must be disciplined in adhering to your trading plan, analyzing the daily data to discover signals and taking these signals. By its very nature, this software approach lends itself to an inconsistently applied decision-making process.

There is a natural human tendency to hopscotch from one trading plan to another, particularly following a losing period, without historically testing a new plan before adopting it for use in the marketplace in real time. It also is difficult to be systematic and precise in analyzing daily data, resulting in incorrect trading signals.

Finally, due to this window of inconsistency and personal judgment factors which enter into the decision-making process, a trader may, during times of stress, selectively choose to execute signals, further undermining the validity of a trading plan.

Because you can link together many entrees as part of your trading plan, the processing time of analysis software typically is very slow. Depending on the number of contracts followed and studies performed on each, it could take the computer up to several hours to complete a daily update before you would be able to analyze the data to arrive at a trading decision. Even with the auto-run capability, you have to change data and system disks as the contracts are being updated.

To trade a fully diversified portfolio, you should have the ability to track at least 60 different contracts. This would allow you to perform intra-commodity spread analysis and compare the signals of at least two contracts per commodity.

Depending on the particular requirements of the analysis software used, this could involve up to three system disks and as many as six data disks. This shuffling of floppy disks is perhaps the most annoying aspect of using analysis software.

After a while, you are forced to make tradeoffs by cutting down on the number of contracts followed and/or the number of studies performed. You end up settling on a small subset off the total smorgasbord menu, defeating the whole basis for employing analysis software in the first place.
By design, analysis software also does not lend itself to inclusion of a valid history tester in the software package. “Paper trading” a proposed plan, “guesstimating” a plan’s performance on several performance measures or testing performance over only a limited time span are not substitutes for adequate testing.

Without a well-designed and statistically valid history tester which provides a comprehensive statistical profile over an extended trading history of a specific commodity, you have no objective way to assess the performance of your customized plan.

The smorgasbord approach made possible by the advent of microcomputers in the late 1970s was definitely an advancement over the hand-held calculator in crunching numbers. Using pre-determined studies chained together with an auto-run capability, the microcomputer certainly can crunch numbers faster and more accurately than could ever be done by hand.

But analysis software still leaves you with raw data as output, and you must personally sift through this data, analyze it according to your customized trading plan and formulate trading signals. At best, you have only a limited ability to historically test the profitability and downside risks of your plan.

All of these tasks can be programmed into the computer, but analysis software falls short in this area. In effect, analysis software is simply not 100% computerized and no longer represents the state-of-the-art in commodity trading software.

System software
System or “black box” software makes more effective use of the computer than analysis software by using mathematical “logic blocks.” These blocks systematically operate on the crunched numbers to generate consistent trading signals. Processing time is usually much faster than analysis software because number crunching is limited to pre-determined studies, although floppy disk swapping still must be done.

But you also make sacrifices with system software:

Many systems are limited for use on one or a few commodities. They have been “curve-fitted” to show profits on these particular commodities but have not proven to be profitable across a wide range of commodities.
You may be required to purchase a customized, dedicated computer from the system vendor.
You have little, if any, control or flexibility over the configuration of the logic block or key system parameters. The system typically is “hardcoated,” not allowing custom tailoring to your capitalization, trading style, risk propensity or changing price characteristics of a commodity over time.
The same signals are generated for all traders using the system. Therefore, the vendor must restrict sales to minimize execution problems.
The system usually does not include a history tester which you can use yourself. Instead, profitability statistics are provided as part of the promotional literature. There is little or no way to verify the figures.
The black box approach restricts you from having knowledge of the underlying logic of the system. You have a right to use but not a right to understand. Therefore, you may be unable to develop confidence in the system.
Depending on the programming expertise and system design, the software may still require you to perform analysis on the data before arriving at a trading signal.
Many systems are offered on a one-time-only basis and do not provide adequate user support through revisions and ongoing enhancements.
Claims of profitability are frequently overstated while performance indicators such as drawdown, consecutive losses and other downside risk factors are often downplayed. Misleading claims, while not necessarily a result of overt false statements, are primarily due to omissions of key statistics and inadequate design of the history tester.

History tester key
The history tester itself is a critical issue, whether it is analysis or system software. Conceptual decisions about the actual design of the history tester will have a great effect on whether the test results will be valid.

These definition and programming decisions include how to handle such critical factors as rollovers, lock limit days, entry/exit timing, data base testing and which exact statistical indicators should be incorporated into the design of the history tester itself.

Most commercially available history testers ignore one or more of these critical issues, treating them as if they did not exist and assuming their absence will have no effect on the validity of the test results.

Track records generated by an inadequately designed history tester or one which omits key statistical indicators result in a biased and invalid picture of a trading system. Regardless of profitability claims, such tests are of little value in assessing the true performance of a system.

Conclusions
Both analysis and system trading software with their current hardware requirements have characteristics and limitations which can prevent you from fully realizing your trading performance objectives. Minimum hardware standards need to be redefined gradually to allow orderly upgrading by existing computer users and to allow first-time purchasers to obtain more advanced hardware as it becomes cost effective.

Technical trading software which is directed solely toward floppy disk systems is already becoming obsolete. In response to this, software developers recently have introduced ram disks and stacked floppies as hardware requirements.

However, these are only interim solutions. They do not eliminate floppy disk swapping, and they are not cost effective when compared to the benefits of hard disk technology. For professional traders and serious speculators who are tracking numerous contracts or obtaining data via on-line quote services or data banks, hard disk systems will become essential.

Software standards
The next generation of software must include a sophisticated history tester, designed in accordance with certain minimum standards of statistical validity accepted within the commodity trading field.

The software should be more responsive and interactive with the trader and designed for maximum processing speed. Underlying mathematical logic of the software and its design should be documented fully in the user manual.

The system should be completely computerized, performing all repetitive tasks within the decision-making process. But it should be flexible enough to allow complete control over all key system parameter values, on a commodity specific basis. Then you can tailor the system to your own risk propensity, capitalization and trading style as well as the price characteristics of each commodity.

Ideally, the software would combine the most desirable features of both analysis and system software. It should give the trader the “bottom line” trading signal based on internal number-crunching and mathematical logic block analysis and should also make the crunched numbers available as background information.

It is through the development and availability of this caliber of software that the software industry supporting the commodity field can maintain its credibility and warrant the confidence and respect of traders.

To fail to do so may result in traders becoming disenchanted with computer technology because they feel short-changed and, in some cases, even ripped off. In this atmosphere, skepticism already evident among commodity traders could develop into outright suspicion as each new “holy grail” claim is put forth.