Parsing Messages from Serial Instruments: RS232, RS485, Modubs, Ethernet, Internet
Data Examples | Setting up Parsing Instructions | Troubleshooting
If you have an instrument or device which you can plug into the PC's COM port, chances are you can use the Windmill software to collect its data. This is because Windmill's universal serial driver, COMIML, was designed to be as flexible as possible and accommodate almost any instrument with an RS232 port. These notes are to help you decide the best way to interpret, or parse, the messages from your instrument.
When an instrument sends a message it rarely just contains data. All sorts of extra characters and codes are in there. Here are some examples of messages.
A weighing scale might send:
+ 125.32 g,
A GPS receiver might send:
$GPGLL,5330.12,N,00215.31,W,134531,A
A Parallax BASIC Stamp might send:
+66.4
A spectrophotometer might send:
DR/4000U S/N: 9807U0000905 2.303-APR-04
10:05:56 MULTI-WAVELENGTHGroup 0000 Sample
0002 -0.000 ABSA=K1A1+K2A2
665.0 nm -0.004 ABS K1: 1.0000 750.0 nm -
0.004 ABS K2: -1.0000
Windmill doesn't just store the entire message, superfluous characters and all. It extracts the relevant data from each message, directing each data item to a channel: latitude readings to one channel, longitude to another and so on. This is called Parsing the Message. Each data item can be up to 20 characters long. There is often no need to collect the units of the reading. You can use the SeupIML program to specify the units of each channel.
It's not only message formats that vary. Some instruments need a command prompt from Windmill before they will send data, others initiate data transfer themselves - often regularly sending data. In this case the message can be considered as a continuous stream of data that has a recurring pattern.
Entering Instructions to Extract Data
To set up parsing instructions use the Windmill ComDebug software and click the Parse button.
There are two steps to parsing the data:
- Locate the data in which you are interested
- Extract the data
You can locate the data by:
- Searching for specific characters in the message string. The next action occurs after all the specified characters
- Ignoring characters until one of the ones in which you are interested appears. The next action occurs on any one of the specified characters
- Ignoring a number of characters in the message string.
At first glance methods 1 and 2 seem to do the same job. However, when you search for characters the next action occurs AFTER the searched for characters. When you ignore characters the next action occurs ON the specified character. So if you searched for a + sign you would not be able to extract it, but if you ignored all characters until the + sign you would. Also, a search will look for the entire string specified (eg "abc"), whilst ignoring characters will stop at any of the characters specified (eg "a" or "b" or "c"). Use the NonPrint menu to enter characters like carriage returns.
Any character shown in the Char column as 3 digits is a "non-printing decimal". In this case 009 is a tab character
Be aware that with instruments that constantly send data Windmill will not know where the beginning of the message is. So, using the GPS example above, it's no use ignoring 8 characters and expecting the next value to be a latitude reading. Instead search for a unique string such as $GPGLL.
Once you've located the data you need to extract it. There are several ways to do this, the two most commonly used are:
- Extract until a specified character occurs. The next action occurs on this character.
- Extract the next so many characters.
Extract until means COMIML or ComDebug will extract values until it finds any one of the characters specified. The next command then acts ON this character: it will not be extracted.
For full details of all the extract options, see the ComDebug Help menu.
Using the GPS receiver as an example again
$GPGLL,5330.12,N,00215.31,W,134531,A
The data we might want to record from this string is the latitude and the longitude. These are the 5330.12 and 00215.31 figures respectively. For the latitude channel we might:
Search for $GPGLL, and extract until ,
For the longitude we might:
Search for N, and extract until ,
As another example, the BASIC Stamp might continually send a reading comprising a leading sign character, a trailing
carriage return to mark the end of data and a maximum length of six characters. For example -55.27
Troubleshooting
We have examples of settings to use for a range of instruments.
It is good practice to continue your parsing to the very end of the instrument's reply. If you finish parsing when you have the data you need, but more bytes are still to come, then Windmill might send a further Prompt message whilst the previous reply is still arriving. These final bytes might then be mistaken for the start of the next reply.
It is always best to identify the reply start in some unambiguous way if this is possible. For example if the reply message was "T=123<CR> " then you should always search for "T=" rather than ignore 2 bytes. This is especially important with continuous flow instruments, where the instrument continually sends data without any prompting.
It is always possible that we start looking for a continuous flow message just when the instrument is halfway through sending the message. In this case we will only get the later part of the message. If we can identify the message start then these bytes will be ignored and ComIML will wait for the next complete message assuming that the TimeOut is set to a sufficient value. Sometimes identifying the start is not possible. For example if the message were "123<CR>" where the 123 could be any number and could be a variable length, then you could Extract to <CR> which would work if there was a reply to a prompt. However if this were continuous flow such an action could sometimes return less than all the digits. The best strategy would be to search for <CR> then Extract to <CR>. This would work reliably but only gets data from every other message sent by the instrument.
If you are still having problems get in touch. Please give as much information as possible, including what steps you have taken so far.
Related Topics
Example Settings for Serial Instruments
RS232 Driver Software
Technical Support
RS232 Standard and Trouble-Shooting Com Port Problems
Serial Driver
Com Port Settings
Data Logging from NMEA Devices
