[ILUG] Breaking up an input stream

[Niall O Broin]

On Thu, Jan 10, 2002 at 12:42:59PM -0000, Adrian Flynn meant to write:

> There is a continuous data stream coming in the serial port.
> There is a FF character at the end of each record.
> Ideally, each record should be stored in a separate file for further
> processing (these records are rather large)
> The question is how to recognise the End-Of-Record character and store all
> data from that record in a (uniquely named) file.
> There is unique data within each record which could form the basis for the
> filename.
>
> Any ideas (sed/awk etc)

Seems relatively trivial but I don't think sed or awk are the tools for the
job - but I'm sure somebody will come up with awk for it - I'd be well
impressed if anybody can do this in sed. Perl would be my poison and something
like this might be a good start

#!/usr/bin/perl
while (<>) { # while there's a character to read from STDIN
    if (/\f/) { # check if it's an FF character and if so, file the data
       $filename = substr($buffer, 1, 10); # figure out the filename however
       open FILE, ">$filename"; # open a file for writing with that name
       print FILE $buffer;      # write the buffer to the file
       close FILE;
       $buffer = "";            # clear the buffer
    } else {    # not an FF so we must put it in the buffer
       $buffer .= $_;
    }
}


The above is absolutely not guaranteed to do anything for you whatsoever
apart from compile. Running this with the serial port as STDIN should do
about what you want but it's COMPLETELY without any kind of error checking or
sanity checking and relies on the OS to do all the buffering / handshaking
on the serial port for you (which is what an OS is for, after all, but you
may have more stringent requirements)

If this is intended for production use and if you go with Perl and you end
up with a program which is not at least twice as long as the above then
you've certainly not got enough error checking :-(


Niall