/* Program: extract.c Version: 1.0.29 Date: 06-MAY-2005 Author: David Mathog, Biology Division, Caltech email: mathog@caltech.edu Copyright: 2002,2003,2004,2005 David Mathog and California Institute of Technology (Caltech) Description: Reads a text file from stdin and extracts a range of rows and columns (character positions) and sends them to lcl_stdout. Alternatively, acts on tokens. Alternatively, removes the selected range instead of emitting it. "extract" is a lot simpler to use than awk or perl and sufficient for most column/row extraction tasks. See function emit_help for command line options. Compiles cleanly with: % gcc -Wall -pedantic -ansi -DMAXINFILE=20 -o extract extract.c change -DMAXINFILE=40 to increase the number of input files which may be processed. This code will probably compile with any modern C compiler. Changes: 1.0.29 06-MAY-2005, Fixed bug in -template, uninitialized variable "remwidth" 1.0.28 30-MAR-2005, Added -tr options (remove leading, trailing, both white space) 1.0.27 15-MAR-2005, Fixed a bug in adjuststring affecting rtss, if it was longer than rtds the substitution was mangled. Changed the base IO model to use super_fgets() instead of fgets(). This allows better handling of embedded nulls in input streams. Added -hnr (retain embedded nulls = DEFAULT) -hnd (delete embedded nulls) -hns (substitute embedded nulls) -hnsubs '|' (character to substitute for embedded NULL, defaults to 255) -hnsubs '\254' (alternate form) -crok (retain a CR before LF. Default is to delete it). 1.0.26 22-SEP-2004, Added fileeol,filebol, a string emitted at the end of a file and at the beginning of a file. 1.0.25 03-JUN-2004, Fixed a bug in rtds, removing "-" from "a---b" -> "a-b", because it was removing every other one. Similarly, -rtds '-' -rtss '.' on '---' went to '.-.' 1.0.24 15-JAN-2004, Fixed access violation on linux. Found with njamd - had not affected operation. Made sure all string malloc() have +1 to hold terminator. 1.0.23 11-JAN-2004, -rs was broken. It worked when the field was off the ends but not for "a,b,,d" and retrieve 3rd field, which came back as an empty string. 1.0.22 18-NOV-2003, -ifterm string Terminates an -if state, use instead of an -ifn. -iftermeol -string Append this EOL string at termination of an -if state. -iftermbol -string emit this string immediately on -if block termination (before processed text). This is primarily to handle the case where the start and end lines of the if blocks are the same. Using it a new line or other characters may be injected between blocks. Primarily useful when the if blocks are collapsing multiple rows of text into one. -bol string, -ifbol which are the beginning of line equivalents to -eol,-ifeol. 1.0.21 21-MAY-2003, -unmerge N. Opposite of -merge (sort of). Reads an input line. Tokenizes with -mdl (which defaults to -dl if not specified). For each group of N tokens after the first it creates an "input line" consisting of the first token + N tokens. This is processed as any other input line. N is stored as -N in the "merge" variable. -template N. Consider two files containing lists, one complete, one partial, both in the same order (sorted, or whatever). If an input file (or files) is read in with the complete list as a template any missing lines are replaces with -trs (or -rs, or just a blank line). Not compatible with -eqlen. N is the number of characters which are compared to determine if there is a match. The first file in the -in list is the template. A blank line in either file terminates it. 1.0.20 15-MAY-2003, Fixed bug. pad and fw were not working if the field was empty Added -rcdc (case insensitive version) Added -rtds STRING (replace text delete string) Added -rtss STRING (replace text substitute string) Either remove a matching string or replace it. 1.0.19 08-MAY-2003, Added -merge N (same prefix merge of consecutive lines) Added -mdl STRING (delimiter between merged consecutive lines) Added -rcds STRING (remove characters delete string) Added -rcss STRING (remove characters substitute string) Use -rcds and -rcss like unix "tr" command. rcds alone to remove (or if it starts with !, keep) characters. With rcss matching characters are substituted from that position in the rcss string. This made the input structure in main() very messy :-(. Note that order is: merge from multiple files merge from -merge prefix if actions output Added [start,end[,increment]] to ranges. Increment is optional and can be negative or positive. Note that when going backwards the delimiter comes from the token before. That is, the delimiter between n and n+1 is shown for n,n+1 and n+1,n. Fixed bug (maybe added others?). [\58] was accepted and came out as 8. modified strtok_variant so that \ does not act in first position of a token. Modified behavior so that a range like [3,-1] does not generate an error on a short line. Instead it becomes [2000000000,20000000] which will always be valid but out of range, yielding an empty token or an empty string. 1.0.18 06-MAY-2003, Fixed bug: [-1] wasn't working, required [-1,-1]. 1.0.17 03-MAY-2003, Added multiple comma separated input files and merge functionality. Added -indl 'string' to indicate a string to be placed between each line. Added -eqlen to force input files to all be the same length. Added -n to emit the current line number. Added ability to detect empty string. CAREFUL! tag=NULL means no if, tag='\0' means empty string!!! 1.0.16 25-APR-2003, Added -fp (floating precision) and [ff] (floating format). fp sets the precision for floating point and ff in an [] expression indicates that the value is a floating point number subject to -fw and -fp. ff can be used on the command line as well, in which case ALL [] are floating point. Fixed potential bug [fw-1:] and [pd-1:] were allowed but didn't make sense, now these values must be >=0. 1.0.15 24-APR-2003, Added -eol, replaces \n with eolstring on the ends of lines. Fixed up input so that it can handle a file which ends with an EOF but no EOL on the last line. Note that such a line will be emitted with an EOL unless -eol or -ifeol intervenes. 1.0.14 16-APR-2003, Added fixupdelimit processing to all plain text strings inside -cols, -rs strings, and -dv strings. So now -dv '\n' emits a line feed between tokens and -cols '[1]\t[2]\n[3]' contains tabs and line feeds. Added -cf (first letter upper, rest lower). Unrecognized \xxx strings are not interpreted and do not throw errors. 1.0.13 10-FEB-2003, added -ifeol. If the logic is satisfied EOL is suppressed. This allows the next line to be appended to if line, or whole series of if lines to be appended to each other. 1.0.12 14-JAN-2003, added -ifonly. Without this rows matching -if/-ifn are modified by -cols, but all lines are emitted. With -ifonly only lines matching -if/-ifn are emitted. 1.0.11 14-NOV-2002, added -ifn, which extends the result of the -if test for N more lines. Each line is also tested for the tag so that the extend can be reset. This allows a "tag" to be recognized and that and the next 5 lines processed. 1.0.10 05-JUL-2002, added -if, which provides a limited conditional capability. The general idea is that a very simple pattern can be matched with -if, and otherwise, put another program upstream to introduce a unique "process this" mark. 1.0.9 15-MAY-2002, added -bs, which adds unix escapes (back slashes) to a string. added code so that -dl ' :\t' makes tab a delimiter. Because on many OS's could not enter tab on the command line directly). 1.0.8 31-MAR-2002, fixed bug where -mt -d- worked but -d- -mt didn't. 1.0.7 05-MAR-2002, sanitized [s,e] syntax and removed TAIL prefix. Meaning is now: e=0,s=0 error s>0 column to start in measured from 1 s<0 column to start in measured from the END of the line (-1 = last column) e>0 column to end in measured from 1 e<0 column to start in measured from the END of the line (-1 = last column) Note that it's a VERY bad idea to do 10 columns from left for start and 10 from right for end as depending upon the line length the range may be invalid! Changes sc storage to NOT have -1 offset. Note also that [2,] may give a fatal error if the line has only 1 token. Cleaned up logic for when delim is emitted in cases like [1,3]foo[4,5] 1.0.6 03-MAR-2002, added [TAIL,offset,N] mode. This allows extraction of N characters/tokens offset from the end of a line. Changed -cols parsing so that \ escapes may be used, for instance, to escape :[]. Enabled [tail,o,n] format. Fixed some bugs. Rearranged some code. 1.0.5 28-FEB-2002, added -rs switches. 1.0.4 25-FEB-2002, added generic text based -cols switch to allow column reordering, multiple gapped column selection, and text insertion. Added -xc (max columns) for data structure to hold token pointers. Renamed pad to pd and added fw to handle the absolute width case. 1.0.3 22-FEB-2002, use lcl_strcasecmp because strcasecmp wasn't ANSI standard (at least on older compilers.) Add -in -out flags for OS's that don't have pipes. 1.0.2 22-FEB-2002, added these flags -pad N pad field with N trailing spaces, if negative, pad or truncate to width. -jl/-jr/-jc left/right/center justify, follows pad if that too is specified -cu/-cl upper case/lower case -is in situ mode (extract/remove/in situ) 1.0.1 20-FEB-2002, added -rm flag, which removes the indicated region instead of extracting it. Can be used iteratively or in a pipe to to pull out several columns from a multicolumn input file. 1.0 05-JAN-2002, first release License terms: You may run this program on any platform. You may redistribute the source code of this program subject to the condition that you do not first modify it in any way. You may distribute binary versions of this program so long as they were compiled from unmodified source code. There is no charge for using this software. You may not charge others for the use of this software. Miscellaneous: Credit to Pat Rankin for writing the VMS EXTRACT utility which inspired this program. */ #include #include #include #include /* for toupper, tolower */ #include #include #define EXVERSTRING "1.0.29 06-MAY-2005" #define MYMAXSTRING 16000 #define MYMAXRANGES 8192 /* default number of column/token ranges that may be specified */ #define MYMAXCOL 128 /* maximum terms in a -cols statement if all 8192 columns are specified singly. */ #define MYSHORTSTRING 100 #define MYMAXTEMPLATE 1000 /* longest compared string in a template */ #define MYMAXDV 4 /* longest possible dv string, like '\123' */ #define MYDEFPRECISION 6 /* floating point precision */ #ifndef MAXINFILE #define MAXINFILE (FOPEN_MAX-1) /* because need 1 open for output file */ #endif /* MAXINFILE */ #define NO 0 #define YES 1 #define MAYBE -1 #define TO_END -1 /* for cases like [1,] */ #define IFFOREVER -1 /* extend an -if block until end of file or terminated */ #define FFNONE 0 #define FFFLOAT 1 /* format to %M.Nf */ #define FFEXP 2 /* format to %M.Ne */ #define FFPDEFAULT 4 #define FFSDEFAULT 5 #define JUSTNONE 0 #define JUSTLEFT 1 #define JUSTRIGHT 2 #define JUSTCENTER 3 #define JUSTPDEFAULT 4 #define JUSTSDEFAULT 5 #define TRIMNONE 0 #define TRIMLEFT 1 #define TRIMRIGHT 2 #define TRIMCOMPRESS 4 #define TRIMPDEFAULT 8 #define TRIMSDEFAULT 16 #define CASENONE 0 #define CASEUPPER 1 #define CASELOWER 2 #define CASEFIRST 3 #define CASEPDEFAULT 4 #define CASESDEFAULT 5 #define SLASHNONE 0 #define SLASHUNIX 1 #define SLASHPDEFAULT 2 #define SLASHSDEFAULT 3 #define DELIMNONE 0 #define DELIMTOK 1 #define DELIMVAL 2 #define DELIMPDEFAULT 3 #define DELIMSDEFAULT 3 #define ACTEXTRACT 0 #define ACTREMOVE 1 #define ACTINSITU 2 #define PADNONE 0 #define PADPDEFAULT 1 #define PADSDEFAULT 2 #define PADFIELD 3 #define FPNONE 0 #define FPPDEFAULT 1 #define FPSDEFAULT 2 #define FPFIELD 3 #define RSNONE 0 /* replacement string - for missing fields */ #define RSPDEFAULT 1 #define RSSDEFAULT 2 #define RSVAL 3 #define RCDNONE 0 /* remove character delete string */ #define RCDPDEFAULT 1 #define RCDSDEFAULT 2 #define RCDVAL 3 /* case sensitive match */ #define RCDVALC 4 /* case insensitive match */ #define RCSNONE 0 /* remove character substitute string */ #define RCSPDEFAULT 1 #define RCSSDEFAULT 2 #define RCSVAL 3 #define RTDNONE 0 /* replace text delete string */ #define RTDPDEFAULT 1 #define RTDSDEFAULT 2 #define RTDVAL 3 /* case sensitive match */ #define RTDVALC 4 /* case insensitive match */ #define RTSNONE 0 /* replace text substitute string */ #define RTSPDEFAULT 1 #define RTSSDEFAULT 2 #define RTSVAL 3 #define MODENONE 1 /* program default mode is MODENONE = MODECHAR */ #define MODECHAR 1 #define MODETOK 2 #define MODEPDEFAULT 3 #define MODESDEFAULT 4 #define TAG_NONE 0 /* tag states, stored in ctag[0] */ #define TAG_ANYWHERE 1 #define TAG_FRONT 2 #define TAG_END 4 #define TAG_NOT 8 #define TAG_MASK 7 /* Parse options for strtok_variant. Conservative means no change to input string. Destructive means that changes may be made to that string. */ #define PARSE_DBLQUOTES 1 /* conservative, double quotes may surround delimiters */ #define PARSE_SINGLEDEL 2 /* conservative, each delim defines a token (default, runs define one token) */ #define PARSE_ESCAPES 4 /* destructive, \ escapes characters and is consumed, ie \: -> : */ #define PARSE_STRIPDQUOTES 8 /* with PARSE_DBLQUOTES, returns tokens without the double quotes */ /* options for handling NULL characters in text strings on input*/ #define HNULL_RETAIN -1 /* retain them - default, fastest, but only works correctly if none are present */ #define HNULL_DELETE 0 /* delete all embedded NULL characters */ #define HNULL_SUBS 255 /* default substitution character for embedded null characters Override with -hnsubs */ /* super_fgets() status bits */ #define SFG_EOF 1 /* input terminated by End of File */ #define SFG_EOL 2 /* input terminated by End of line (\n) */ #define SFG_CRLF 4 /* input terminated by CRLF (\r\n) \r remains! */ #define SFG_EMBEDDED_NULL 8 /* embedded NULL characters were read */ #define SFG_BUFFER_FULL 16 /* input buffer full */ #define SFG_READERROR 32 /* unrecoverable read error */ typedef struct is_field FIELDNEXUS; typedef struct is_string_struct STRINGNEXUS; struct is_string_struct { char *string; /* start of the string buffer */ char delimit; /* 0 or the delimiter */ int start; /* start of the substring in the buffer */ int send; /* end of the substring */ int check; /* check line lengths at emission. Yes if for buffer, no for cmd */ }; struct is_field { int iss; /* first iss to ISS to emit */ int issend; /* last iss to emit */ int issinc; /* increment from first to last by this (default to 1)*/ int ttype; /* MODENONE=MODECHAR,MODETOK,MODEP/SDEFAULT */ int pad; /* pad value, used if lpad==PADFIELD, also used for field width */ int lpad; /* PADNONE,PADP/SDEFAULT,PADFIELD (with .pad value) */ int precision; /* floating point precision value */ int lprecision; /* FPNONE,FPP/FPDEAFULT,FPFIELD (with .precision value) */ int ff; /* FFNONE,FFFLOAT,FFEXP,FFP/SDEFAULT */ int justify; /* JUSTNONE,JUSTLEFT,JUSTRIGHT,JUSTCENTER,JUSTP/SDEFAULT */ int trimify; /* BIT FIELD! TRIMNONE,TRIMLEFT,TRIMRIGHT,TRIMCOMPRESS,TRIMP/SDEFAULT */ int casify; /* CASENONE,CASEUPPER,CASEFIRST,CASELOWER,CASEP/SDEFAULT */ int slashify; /* SLASHNONE,SLASHUNIX,SLASHP/SDEFAULT */ char delim; /* used sometimes, depending upon ldelim) */ int ldelim; /* DELIMNONE,DELIMP/SDEFAULT,DELIMTOK (with value from token),DELIMVAL (with .delim value) */ int lrs; /* RSNONE,RSP/SDEFAULT,RSVAL */ int lrcd; /* RCDNONE,RCDP/SDEFAULT,RCDVAL,RCDVALC */ int lrcs; /* RCSNONE,RCSP/SDEFAULT,RCSVAL */ int lrtd; /* RTDNONE,RTDP/SDEFAULT,RTDVAL,RTDVALC */ int lrts; /* RTSNONE,RTSP/SDEFAULT,RTSVAL */ char *ersstring; /* holds the RS string, if there is one */ char *rcd; /* holds the rcd string, if there is one */ char *rcs; /* holds the rcs string, if there is one */ char *rtd; /* holds the rtd string, if there is one */ char *rts; /* holds the rts string, if there is one */ }; /* function prototypes */ void add_op(char *cmd, int start, int length, FIELDNEXUS *fieldlist, STRINGNEXUS *isslist, int *dynamic, int *fieldcount, int *maketokens, char *buffer, FIELDNEXUS *gblparams); void adjuststring(char *buffer,int blen,char *cptr,int clen, int shiftval); void caseit(char *string,int casify); void classify_op(char *cmd, int *sc, int *nc, int *inc, FIELDNEXUS *lclparams); void dumpcommands(FIELDNEXUS *fieldlist,STRINGNEXUS *isslist, int dynamic, int tcount, int fieldcount, int linelen, char *cols); void emit_help(void); void emit_output(char *piece, FIELDNEXUS *fieldlist, STRINGNEXUS *isslist, int tcount,int dynamic,int fieldcount,int maxwidth, int linelen, char *cols, int ifeol, int ifbol, char *eol, char *bol, int linenumber, int linecount); void ffit(char *string,int ff, int width, int precision, int maxwidth); void fixupdelimit(char *delimit); void fixuptag(char *tag, int *tagstate); int getnextline(char *buffer,int maxwidth,FILE **lcl_stdin,int infiles, char *indl, int samelines, int template, int handlenull, int crok); void get_range(char *string, int length, int *sc, int *nc, int *inc); void emit_help(void); void emit_help_cols(void); void emit_help_examples(void); void init_process_type(char *cmd, char * buffer, FIELDNEXUS *fieldlist, STRINGNEXUS *isslist, int *dynamic, int *fieldcount,int *maketokens, FIELDNEXUS *gblparams); void insane(char *string); void insane2(char *s1, char *s2); void justit(char *string,int justify); int lcl_strcasecmp(char *s1, char *s2); int lcl_strcmprange(char *s1, char *s2, int start, int end, int usecase); int lstrcasecmp(char *s1, int len1, char *s2, int len2); int lstrtol(char *string,int length, long *ival); void makealltokens(char *buffer, char *delimit, STRINGNEXUS *isslist, int parse_options, char *emptystring, int dynamic, int *etcount); int nextpartialbuffer(char *pbuffer, char **first, int *flen, char **string, int parse_options, int ntokens, char *mdl); void padit(char *string,int pad,int maxwidth); void process_command_line_args(int argc,char **argv, int *maxwidth, int *sc,int *ec, int *sr,int *er, int *rm, int *parse_options, int *all, FIELDNEXUS *gblparams, int *xc, int *dbg, int *tagstate, int *tagtermstate, int *ifn, int *ifonly, int *ifeol, int *ifbol, char **eol, char **bol, char **cols, char **delimit, char **tag, char **tagterm, char **iftermeol, char **iftermbol, char **iffileeol, char **iffilebol, char **indl, int *samelines, int *linenumber, int *merge, char **mdl, int *template, int *handlenull, int *crok); void rcit(char *string,char *rstring,char *sstring, int rcdmode); void realends(int *rstart, int *rend, int lstart, int lend, int length); void regular_to_cols(char **cols, char *buffer,int sc,int ec, int rm); char * strtok_variant(char *instring,const char *delim, int parse_options, char *empty, char *thedelim, int *start, int *length); void rtit(char *string,int blen, char *rstring,char *sstring, int rtdmode); void setuchar(unsigned char *val,int *numarg,int argc,char **argv,char * label); void setiposnumeric(int *val,int *numarg,int argc,char **argv,char * label); void setinumeric(int *val,int *numarg,int argc,char **argv,char * label); void setnonzeronumeric(int *val,int *numarg,int argc,char **argv,char * label); void setonechar(char *val,int *numarg,int argc,char **argv,char * label); void setstring(char **val,int *numarg,int argc,char **argv,char * label); void slashit(char *string,int slashify, int maxwidth); void slideleftone(char *string,int start,int *lastchar); char *stristr(char *String, char *Pattern); int super_fgets(char *string, size_t size, FILE *stream, size_t *cterm, int enchar); int tagcheck(char *buffer, char *tag, int tagstate, int taglength); void trimit(char *string,int trimify); /* globals */ FILE *lcl_stdin[MAXINFILE]; int infiles=1; FILE *lcl_stdout; /* functions */ void insane(char *string){ (void) fprintf(stderr,"%s\n",string); exit(EXIT_FAILURE); } void insane2(char *s1, char *s2){ (void) fprintf(stderr,"%s [%s]\n",s1,s2); exit(EXIT_FAILURE); } void emit_help(void){ if(lcl_stdout==NULL)lcl_stdout=stdout; (void) fprintf(lcl_stdout,"extract command summary:\n\n"); (void) fprintf(lcl_stdout,"This program extracts data from lines of text, either:\n"); (void) fprintf(lcl_stdout," Method 1: blocks of text by character column ranges\n"); (void) fprintf(lcl_stdout," Method 2: specified text token(s)\n"); (void) fprintf(lcl_stdout," Processing may be restricted to a range of text rows.\n"); (void) fprintf(lcl_stdout," Data is read/written from/to stdin/stdout or input/output files.\n\n"); (void) fprintf(lcl_stdout,"Command line options are:\n\n"); (void) fprintf(lcl_stdout," -in file1,file2,file3...file%d input files (Default = stdin)\n",MAXINFILE); (void) fprintf(lcl_stdout," If more than one input file paste lines side by side\n"); (void) fprintf(lcl_stdout," Use \"-\" to read one stream from stdin.\n"); (void) fprintf(lcl_stdout," -out file output file (Default = stdout)\n"); (void) fprintf(lcl_stdout," -cols FMT fine control of field extraction and line formatting.\n"); (void) fprintf(lcl_stdout," See -hcols and -hexamples. (Default = use -sc etc. for format)\n"); (void) fprintf(lcl_stdout," -wl N widest line (Default = 16000 characters)\n"); (void) fprintf(lcl_stdout," -sc N start column (1 is first and default)\n"); (void) fprintf(lcl_stdout," -ec N end column (<=wl, Default = wl)\n"); (void) fprintf(lcl_stdout," -nc N number of columns (Specify -ec or -nc but not both!)\n"); (void) fprintf(lcl_stdout," -sr N start row (1 is first and default)\n"); (void) fprintf(lcl_stdout," -er N end row (Default = last row of file)\n"); (void) fprintf(lcl_stdout," -nr N number of rows (Specify -er or -nr but not both!)\n"); (void) fprintf(lcl_stdout," -if tag\n"); (void) fprintf(lcl_stdout," Modify row in range using -cols only if it contains the text tag.\n"); (void) fprintf(lcl_stdout," Formats: [!][^]tag[$], ! = not in, ^ = front, $ = end \n"); (void) fprintf(lcl_stdout," -ifn N extend result of -if N more lines or until EOF.\n"); (void) fprintf(lcl_stdout," -ifterm tag extend result of -if until -ifterm tag matches or EOF\n"); (void) fprintf(lcl_stdout," -iftermeol STR \n"); (void) fprintf(lcl_stdout," emit STR after the last line of an -if block\n"); (void) fprintf(lcl_stdout," -iftermbol STR \n"); (void) fprintf(lcl_stdout," emit STR before the last line of an -if block\n"); (void) fprintf(lcl_stdout," -ifonly only lines in an -if block are emitted\n"); (void) fprintf(lcl_stdout," -ifeol omit EOL on lines within an -if block (does not effect iftermeol)\n"); (void) fprintf(lcl_stdout," -ifbol omit BOL on lines within an -if block (does not effect iftermbol)\n"); (void) fprintf(lcl_stdout," -dl STR token delimiters (Default = space,colon,tab)\n"); (void) fprintf(lcl_stdout," -fileeol STR \n"); (void) fprintf(lcl_stdout," emit STR after the last line of a file (default = none)\n"); (void) fprintf(lcl_stdout," -filebol STR \n"); (void) fprintf(lcl_stdout," emit STR before the first line of a file (default = none)\n"); (void) fprintf(lcl_stdout," -eol STR terminate lines with string (Default = end with \\n)\n"); (void) fprintf(lcl_stdout," -bol STR prefix lines with string (Default = no prefix )\n"); (void) fprintf(lcl_stdout," -indl STR string to place between input from multiple input files (Default=none)\n"); (void) fprintf(lcl_stdout," The delimiter string usually must be quoted.\n"); (void) fprintf(lcl_stdout," -mdl STR string to place between input from multiple merged rows (Default=none)\n"); (void) fprintf(lcl_stdout," -rs STR replacement string emitted when a column field is empty\n"); (void) fprintf(lcl_stdout," (Default = do not replace empty fields)\n"); (void) fprintf(lcl_stdout," -rcds STR delete characters found in STR. (Keep if str is \"!characters\").\n"); (void) fprintf(lcl_stdout," (Default = allow all characters)\n"); (void) fprintf(lcl_stdout," -rcdc STR case insensitive variant of -rcds\n"); (void) fprintf(lcl_stdout," -rcss STR if character j in -rcds STR matches replace it with character j from -rcss STR\n"); (void) fprintf(lcl_stdout," (Default = no substitutions)\n"); (void) fprintf(lcl_stdout," -rtds STR delete STR from text (case sensitive).(Default = no action)\n"); (void) fprintf(lcl_stdout," -rtdc STR delete STR from text (case insensitive).(Default = no action)\n"); (void) fprintf(lcl_stdout," -rtss STR replace text removed by -rtds with STR (Default = no substitutions)\n"); (void) fprintf(lcl_stdout," -all emit lines beyond row limits.\n"); (void) fprintf(lcl_stdout," (Default = do not emit)\n"); (void) fprintf(lcl_stdout," -mc extract characters (Default mode)\n"); (void) fprintf(lcl_stdout," -mt extract tokens (Default = extract characters)\n"); (void) fprintf(lcl_stdout," With -mt -sc/-ec/-nc refer to token numbers\n"); (void) fprintf(lcl_stdout," -eqlen require multiple input files to have exactly the same number of lines\n"); (void) fprintf(lcl_stdout," -s each delimiter emits a token\n"); (void) fprintf(lcl_stdout," Use -s with delimited data as from a spreadsheet.\n"); (void) fprintf(lcl_stdout," (Default = delimiter runs emit one token.)\n"); (void) fprintf(lcl_stdout," -dq ignore delimiters within double quoted strings\n"); (void) fprintf(lcl_stdout," (Default = recognize all delimiters)\n"); (void) fprintf(lcl_stdout," -dqs like -dq, but surrounding double quotes are stripped\n"); (void) fprintf(lcl_stdout," (Default = recognize all delimiters)\n"); (void) fprintf(lcl_stdout," -dt emit all but last token with first delimit character (Default)\n"); (void) fprintf(lcl_stdout," -dv C emit all but last token followed by delimit character C\n"); (void) fprintf(lcl_stdout," (Default = use actual delimiter)\n"); (void) fprintf(lcl_stdout," -d- emit tokens without delimit character.\n"); (void) fprintf(lcl_stdout," (Default = emit with actual delimiter)\n"); (void) fprintf(lcl_stdout," -rm remove selected columns/tokens \n"); (void) fprintf(lcl_stdout," (Default = emit selected)\n"); (void) fprintf(lcl_stdout," -is modify selected columns/tokens and emit all \n"); (void) fprintf(lcl_stdout," (Default = emit selected)\n"); (void) fprintf(lcl_stdout," -crok retain a CR immediately before an EOL (Default, delete these)\n"); (void) fprintf(lcl_stdout," -hnr retain embedded null characters (Default, fatal if nulls are found)\n"); (void) fprintf(lcl_stdout," -hnd delete embedded null characters \n"); (void) fprintf(lcl_stdout," -hns substitute embedded null characters with \"\\%d\" \n",HNULL_SUBS); (void) fprintf(lcl_stdout," -hnsubs CHAR\n"); (void) fprintf(lcl_stdout," substitute embedded null characters with CHAR\n"); (void) fprintf(lcl_stdout," -xc N maximum columns allowed (Default = %d)\n",MYMAXRANGES); (void) fprintf(lcl_stdout," -pd N append N spaces to column (Default = no pad)\n"); (void) fprintf(lcl_stdout," -fw N make field N spaces wide (Default = original width)\n"); (void) fprintf(lcl_stdout," -fp N make floating point precision N spaces wide (Default = 6)\n"); (void) fprintf(lcl_stdout," -fff/ffe format all fields as numeric float/exponent formats (Default = text fields)\n"); (void) fprintf(lcl_stdout," -jl/jr/jc left/right/center justify (Default = no justify)\n"); (void) fprintf(lcl_stdout," -trl/trr/trb trim off whitespace left side/right side/both sides (Default = no trim)\n"); (void) fprintf(lcl_stdout," -cu/cf/cl upper case/first letter upper & rest lower/lower case (Default = case unchanged)\n"); (void) fprintf(lcl_stdout," -bs place a back slash (unix escape) before special characters\n"); (void) fprintf(lcl_stdout," -merge N merge consecutive rows starting with the same N characters, separated by -mdl STR\n"); (void) fprintf(lcl_stdout," -unmerge N opposite of merge, works only with tokens delimited by -mdl STR\n"); (void) fprintf(lcl_stdout," Emit lines: first token + N tokens until line is consumed.\n"); (void) fprintf(lcl_stdout," -template N Fill in missing records in a file using a template file.\n"); (void) fprintf(lcl_stdout," For each row the first N characters are compared. The rows in the two\n"); (void) fprintf(lcl_stdout," files must be in the same order. Use \"-in template,file\". Missing \n"); (void) fprintf(lcl_stdout," rows from file are replaced by the first N characters from template plus -indl STR.\n"); (void) fprintf(lcl_stdout," In this mode only a blank line is equivalent to an EOF.\n"); (void) fprintf(lcl_stdout," -n emit input_line_number: before each output line.\n"); (void) fprintf(lcl_stdout," -dbg emit state and parsing information\n"); (void) fprintf(lcl_stdout," -h print this help message (also -help --h --help -? --?)\n"); (void) fprintf(lcl_stdout," -hcols show the (many) -cols options\n"); (void) fprintf(lcl_stdout," -hexamples show examples\n"); (void) fprintf(lcl_stdout," -i emit version, copyright, license and contact information\n\n"); (void) fprintf(lcl_stdout,"Special characters in strings: \\t is tab, \\n is LF, \\r is CR,\\\\ is \\ \n"); (void) fprintf(lcl_stdout," \\19 is the character with value 19 decimal.\n"); exit(EXIT_FAILURE); } void emit_help_cols(void){ if(lcl_stdout==NULL)lcl_stdout=stdout; (void) fprintf(lcl_stdout,"extract -cols options:\n\n"); (void) fprintf(lcl_stdout," -cols FORMAT allows fine grained control over multiple\n"); (void) fprintf(lcl_stdout," columns. For instance: \"STR1[cu:1,5]STR2[mt:5]\"\n"); (void) fprintf(lcl_stdout," STR (text outside of []) may contain any character. Use \\[, \\] or\n"); (void) fprintf(lcl_stdout," [[, ]] for a square bracket as a string character instead of a field delimiter. \n"); (void) fprintf(lcl_stdout," [] is a column or token range. It may contain escaped characters\n"); (void) fprintf(lcl_stdout," including escaped square brackets.\n"); (void) fprintf(lcl_stdout," In all cases:\n"); (void) fprintf(lcl_stdout," + set_as = match command line specifications\n"); (void) fprintf(lcl_stdout," p default = match program defaults\n"); (void) fprintf(lcl_stdout," - disable = disable option\n"); (void) fprintf(lcl_stdout," Each [] column or token operator contains one or more of the following.\n"); (void) fprintf(lcl_stdout," -/+/p in first position - set all fields to indicated state\n"); (void) fprintf(lcl_stdout," mt/mc/m-/mp/m+ modifies -mt -mc\n"); (void) fprintf(lcl_stdout," token fields/character fields/disable/default/set_as\n"); (void) fprintf(lcl_stdout," jl/jr/jc/j-/jp/j+ modifies -j*\n"); (void) fprintf(lcl_stdout," justify left/right/center/disable/default/set_as\n"); (void) fprintf(lcl_stdout," trl/trr/trb/tr-/trp/tr+ modifies -tr*\n"); (void) fprintf(lcl_stdout," trims left/right/both/disable/default/set_as\n"); (void) fprintf(lcl_stdout," cu/cl/cf/c-/cp/c+ modifies -c*\n"); (void) fprintf(lcl_stdout," case upper/lower/first upper/disable/default/set_as\n"); (void) fprintf(lcl_stdout," bs/b-/bp/b+ modifies -bs\n"); (void) fprintf(lcl_stdout," back slash apply/disable/default/set_as\n"); (void) fprintf(lcl_stdout," dt/dvN/d-/dp/d+ modifies -d*\n"); (void) fprintf(lcl_stdout," emit delimit from token/with char N/disable/default/set_as\n"); (void) fprintf(lcl_stdout," pd###/pd-/pdp/pd+ modifies -pd/-fw\n"); (void) fprintf(lcl_stdout," pad with ### spaces/disable/default/set_as\n"); (void) fprintf(lcl_stdout," fw###/fw-/fwp/fw+ modifies -fw/-pd\n"); (void) fprintf(lcl_stdout," field width to ### spaces/disable/default/set_as\n"); (void) fprintf(lcl_stdout," fp###/fp-/fpp/fp+ modifies -fp\n"); (void) fprintf(lcl_stdout," floating precision to ### spaces/disable/default/set_as\n"); (void) fprintf(lcl_stdout," fff/ffe/ff-/ffp/ff+ modifies -ff*\n"); (void) fprintf(lcl_stdout," floating format float/exp/disable/default/set_as\n"); (void) fprintf(lcl_stdout," rssSTR/rs-/rsp/rs+ modifies -rs\n"); (void) fprintf(lcl_stdout," rs is STR/from -rs/disable/default/set_as\n"); (void) fprintf(lcl_stdout," rcdsSTR/rcdcSTR/rcd-/rcdp/rcd+ modifies -rcds/-rcdc\n"); (void) fprintf(lcl_stdout," rcds is STR/case insensitive STR/from -rcds/disable/default/set_as\n"); (void) fprintf(lcl_stdout," rcssSTR/rcs-/rcsp/rcs+ modifies -rcss\n"); (void) fprintf(lcl_stdout," rcss is STR/from -rcss/disable/default/set_as\n"); (void) fprintf(lcl_stdout," rtdsSTR/rtd-/rtdp/rtd+ modifies -rtds\n"); (void) fprintf(lcl_stdout," rtds is STR/case insensitive STR/from -rtds/disable/default/set_as\n"); (void) fprintf(lcl_stdout," rtssSTR/rtscSTR/rts-/rtsp/rts+ modifies -rtss/-rtsc\n"); (void) fprintf(lcl_stdout," rtss is STR/from -rtss/disable/default/set_as\n"); (void) fprintf(lcl_stdout," Followed by the required range value. Range formats are:\n"); (void) fprintf(lcl_stdout," [c] [s,e] [s,] [,e] for single column, column\n"); (void) fprintf(lcl_stdout," range, and open ended ranges. Positive values: 1 is first column/token,\n"); (void) fprintf(lcl_stdout," Negative values -1 is last column/token, -2 second from last. If s is\n"); (void) fprintf(lcl_stdout," negative e must be positive and is a count of columns/tokens to emit.\n"); (void) fprintf(lcl_stdout," For tokens [s,e,i] is allowed, where i is a positive or negative increment value.\n"); (void) fprintf(lcl_stdout," In column mode increment values are ignored.\n"); (void) fprintf(lcl_stdout," Arbitrary column order is allowed and columns may be repeated.\n"); exit(EXIT_FAILURE); } void emit_help_examples(void){ if(lcl_stdout==NULL)lcl_stdout=stdout; (void) fprintf(lcl_stdout,"extract examples:\n\n"); (void) fprintf(lcl_stdout," %% extract (print help message)\n"); (void) fprintf(lcl_stdout," %% extract -h (print help message)\n"); (void) fprintf(lcl_stdout," %% extract -sc 50 fout (extract characters 50 to end of row\n"); (void) fprintf(lcl_stdout," for every line in fin and write to fout)\n"); (void) fprintf(lcl_stdout," %% extract -sr 4 -sc 5 -ec 10 fout\n"); (void) fprintf(lcl_stdout," (extract characters 5-10 from rows 4\n"); (void) fprintf(lcl_stdout," to end of fin and write to fout)\n"); (void) fprintf(lcl_stdout," %% extract -sc 5 -nc 10 c2)return 1; if(c1 == 0)return 0; /*c2 also is 0 in this case */ } } /* compare between start and end only. Have to be careful about sizes here because one or other or both could terminate before the comparison region returns: 1 if s1> s2 0 if s1==s2 -1 if s1< s2 -2 if both terminated before comparison range */ int lcl_strcmprange(char *s1, char *s2, int start, int end, int usecase){ int c1,ok1; int c2,ok2; int i; if(end<0)return -2; if(start<0)return -2; if(end=start) && (i<=end)){ if(c1 < c2)return -1; if(c1 > c2)return 1; } } else if(ok1){ if(i>=start)return 1; /*s2 terminated, s1 did not, s1 is bigger*/ } else if(ok2){ if(i>=start)return -1; /*s1 terminated, s2 did not, s2 is bigger*/ } else { if(i>=start)return 0; /* both terminated within comparison region, equivalent */ return -2; /* both terminated before comparison, no comparison possible */ } } return 0; /* over range they are identical*/ } /* case insensitive comparison for lstr, which are strings consisting of a pointer and a length but no 0 terminator. For a match the lengths must be equal as well. If prefixes are identical the shorter string is lexically smaller than the longer. */ int lstrcasecmp(char *s1, int len1, char *s2, int len2){ int c1,c2; int idx; for(idx=1; ;idx++){ if(idx > len1){ if(idx > len2){ return 0; /* identical, idx went past both at the same time */ } else { return -1; /* identical prefix but s1 is longer */ } } else { if(idx > len2){ return 1; /* identical prefix but s2 is longer */ } else { c1=toupper(s1[idx-1]); c2=toupper(s2[idx-1]); if(c1 < c2)return -1; if(c1 > c2)return 1; } } } } /* tokenize the input line. Store these into the dynamic pointers area of isslist. Note that ALL pointers are to buffer, with varying start/length. */ void makealltokens(char *buffer, char *delimit, STRINGNEXUS *isslist, int parse_options, char *emptystring, int dynamic, int *etcount){ int tcount,length,start; char *atoken; char thedelim; tcount=0; atoken=strtok_variant(buffer,delimit,parse_options,emptystring,&thedelim,&start,&length); while(atoken != NULL){ if(length > 0){ isslist[dynamic + tcount].string = buffer; isslist[dynamic + tcount].start = start+1; isslist[dynamic + tcount].send = start+length; } else { isslist[dynamic + tcount].string = NULL; isslist[dynamic + tcount].start = 1; isslist[dynamic + tcount].send = 1; } isslist[dynamic + tcount].check = YES; isslist[dynamic + tcount].delimit = thedelim; tcount++; atoken=strtok_variant(NULL,delimit,parse_options,emptystring,&thedelim,&start,&length); } *etcount=tcount; }/* makealltokens */ void slideleftone(char *string,int start,int *lastchar){ int i; for(i=start ; i<=*lastchar ; i++){ string[i] = string[i+1]; } (*lastchar)--; } /* This strtok implementation can act like the regular strtok() or it can perform certain other processing modes. These are defined by bits in the parse_options parameter. If none are set, it acts like strtok, otherwise: PARSE_DBLQUOTES Treat "stringDELIMITstringDELIMIT"DELIMIT as one token including the end double quotes. (Internal delimiters are ignored.) PARSE_SINGLEDEL Treat a series DELIMIT DELIMIT as DELIMIT EMPTYSTRING DELIMIT, the latter form is required for properly handling tab delimited data when data fields may be empty. PARSE_ESCAPES \ escapes characters but is NOT consumed, ie \: -> \: \ may escape a delimiter \ may not escape the final '\0', ie, \EOL is the same as EOL. PARSE_STRIPDQUOTES When used with PARSE_DBLQUOTES causes the token to be returned without the surrounding double quotes. It returns a pointer to the empty string. This is modified from mpcf_strtok in the miniproc program. It lacks the single quote processing and the & processing. Double quotes are returned here WITH the quotes attached in dquotes mode. Also returns thedelim = the delimiter, or first delimiter in a run, that terminated a token. Also returns start = offset in buffer to first byte of token (ie, &buffer[start]==atoken) Also returns length = number of bytes in atoken. Atoken is NOT a null terminated string!!!*/ char * strtok_variant(char *instring,const char *delim, int parse_options, char *empty, char *thedelim, int *start, int *length){ static char *holdstring; static int ftoken, etoken, dqetoken, endstring, firstcase; int cpnt; enum sstates {NEXTDELIM,NEXTDQUOTE,NEXTDELIMDQ,NEXTREGULAR,ENDSEARCH}; enum sstates searchstate; enum vdquote {ISVALID,ISINVALID}; enum vdquote validdquote; if(instring != NULL){ /* new string */ endstring = strlen(instring) - 1; holdstring = instring; /* write on the actual input string, not a copy */ ftoken=0; etoken=0; firstcase=1; } else{ /* look for next token in existing string */ etoken++; /* last call would have left this on a delimiter */ ftoken=etoken; firstcase=0; if(etoken > endstring)return NULL; } /* this is the part that does the search an exit path is left from this loop to a return=NULL outside so that some compilers (DECC) won't complain about there not being a final return statement */ for( searchstate = NEXTREGULAR; searchstate != ENDSEARCH ; etoken++){ if(searchstate != NEXTREGULAR){firstcase=0;} /* escaping has highest priority. Normally if a character is escaped everything starting with it and going right slides left one and all further processing is bypassed (to avoid interpreting the escaped character). However, if the escape as the very last character that would escape the EOL character, so in that one instance processing continues as it would for an unescaped EOL */ if(parse_options & PARSE_ESCAPES && ftoken != etoken){ if(holdstring[etoken]=='\\'){ if(etoken == endstring)continue; /* \EOL, so \ cannot escape anything */ etoken++; if(etoken == endstring)continue; /* \XEOL, so X is last character and EOL terminates */ etoken++; /* \XEOLY... , so Y or later could terminate token, fall through */ } } if(etoken > endstring){ /* always check first to see if search has extended past string end */ switch (searchstate){ case ENDSEARCH: /* gcc wants ENDSEARCH case, but from for()logic case is never used */ case NEXTREGULAR: /* hit end of string, no tokens found */ searchstate=ENDSEARCH; break; case NEXTDELIM: /* end of token is end of string */ *thedelim='\0'; *length=etoken-ftoken; *start=ftoken; return &holdstring[ftoken]; case NEXTDELIMDQ: /* handle double quote, the potential double quoted string MUST be followed by a delimiter or an EOL */ etoken=dqetoken; etoken++; validdquote=ISINVALID; if(etoken == endstring + 1){ /* EOL case */ validdquote=ISVALID; } else{ for(cpnt=0; delim[cpnt] != '\0';cpnt++){ if(holdstring[etoken] == delim[cpnt]){ /* NON-EOL case */ validdquote=ISVALID; break; } } } if(validdquote == ISVALID){ *thedelim=holdstring[etoken]; *length=etoken-ftoken; *start=ftoken; return &holdstring[ftoken]; } else{ insane("fatal error, unmatched double quotes"); } case NEXTDQUOTE: /* unmatched double quotes - bad */ insane("fatal error, unmatched double quotes"); } } else{ switch (searchstate){ case ENDSEARCH: /* gcc wants ENDSEARCH case, but from for()logic case is never used */ break; case NEXTREGULAR: /* is the current character NOT a delimiter? */ switch(holdstring[etoken]){ case '"': /* start of a double quote delimited token */ if(parse_options & PARSE_DBLQUOTES){ searchstate = NEXTDQUOTE; ftoken = etoken; /* double quotes are part of the token */ break; } /* DO NOT SEPARATE the " case from default: or the noquotes logic breaks! */ default: searchstate=NEXTDELIM; ftoken = etoken; for(cpnt=0; delim[cpnt] != '\0';cpnt++){ if(holdstring[etoken] == delim[cpnt]){ searchstate=NEXTREGULAR; } } if(searchstate!=NEXTDELIM){ if(parse_options & PARSE_SINGLEDEL){ /* 1 delimiter = full token delimit in this mode EACH is a delimit*/ ftoken=etoken; *thedelim=holdstring[etoken]; *length=0; *start=0; return empty; } } } break; case NEXTDELIM: /* see if the current character is a delimiter */ for(cpnt=0; delim[cpnt] != '\0';cpnt++){ if(holdstring[etoken] == delim[cpnt]){ /* a token has been found */ *thedelim=holdstring[etoken]; *length=etoken-ftoken; *start=ftoken; return &holdstring[ftoken]; } } break; case NEXTDELIMDQ: /* after a "foo foo", this must be a delimiter */ for(cpnt=0; delim[cpnt] != '\0';cpnt++){ if(holdstring[etoken] == delim[cpnt]){ /* a token has been found */ if(parse_options & PARSE_STRIPDQUOTES){ *thedelim=holdstring[etoken]; *length=etoken-ftoken-2; *start=ftoken+1; return &holdstring[ftoken+1]; } else { *thedelim=holdstring[etoken]; *length=etoken-ftoken; *start=ftoken; return &holdstring[ftoken]; } } } insane("fatal error, double quotes followed by more text - should be EOL or delimiter"); break; case NEXTDQUOTE: /* see if the current character is a quote */ if(holdstring[etoken] == '"'){ /* a valid token has been found, may not be final one though */ searchstate = NEXTDELIMDQ; /* indicates that at least one valid token was found */ dqetoken = etoken; } break; } } } *thedelim='\0'; *length=0; *start=0; return NULL; } /* in buffer move everything to the right of "cptr" by shiftval positions. If shiftval is negative slide left. blen is the length of the buffer and clen the length of whatever was right of cptr. Error if stuff gets shoved off either end. */ void adjuststring(char *buffer,int blen,char *cptr,int clen, int shiftval){ char *bend,*cend; char *from,*to; if(shiftval==0)return; bend=buffer + sizeof(char)*(blen); /* bend is on the last character of the big buffer*/ cend=cptr + sizeof(char)*strlen(cptr) + shiftval; if(shiftval>0){ to=cend; /* first character moved goes here*/ from=to - sizeof(char)*shiftval; /* first character moved from here */ if(to>bend)insane("extract: fatal error: string adjust ran off right side of buffer"); for(;from>=cptr;to--,from--){ *to=*from; } } else { /* shiftval < 0 */ to=cptr + sizeof(char)*shiftval; from=cptr; if(to= 0){ /* valid numeric escape*/ *optr = temp; cptr--; /*so that loop increment places it correctly */ } else { /* invalid numeric escape */ cptr=holdptr; cptr--; } } else { /* generic escape, next character goes through no matter what it is */ *optr=*cptr; } } else { *optr=*cptr; } } *optr='\0'; } /* interprets leading ^ as front, trailing $ as end, leading \^ as just ^ and trailing \$ as just $. Interpret leading ! as "NOT". */ void fixuptag(char *tag, int *tagstate){ char *cptr; char *optr; int temp; for(optr=cptr=tag;*cptr!='\0';cptr++){ if(*cptr == '\\'){ cptr++; if(*cptr=='t'){ *optr='\t'; } else if(*cptr == '\\' || *cptr == '^' || *cptr == '$' || *cptr == '!') { *optr=*cptr; } else { /* numeric */ for(temp=0;isdigit((unsigned int) *cptr);cptr++){ temp= 10*temp + (*cptr - '0'); } if(temp > 255 || temp == 0){ insane2("extract, fatal error, -if contains invalid escaped char value",tag); } *optr = temp; cptr--; /*so that loop increment places it correctly */ } optr++; } else { if(*cptr == '!'){ if(optr != tag)insane("extract, fatal error, -if contains unescaped,out of order \"!\""); *tagstate |= TAG_NOT; } else if(*cptr == '^'){ if(optr != tag)insane("extract, fatal error, -if contains unescaped,out of order \"^\""); *tagstate |= TAG_FRONT; } else if(*cptr == '$'){ cptr++; if(*cptr != '\0')insane("extract, fatal error, -if contains unescaped,out of order \"$\""); *tagstate |= TAG_END; break; /* force exit */ } else { *optr = *cptr; optr++; } } } *optr='\0'; if(optr==tag){ /* tag is an empty string, use for matching empty lines */ *tag='\0'; } if(! (*tagstate & TAG_MASK)){ *tagstate |=TAG_ANYWHERE; } } void setiposnumeric(int *val,int *numarg,int argc,char **argv,char * label){ (*numarg)++; if( ( *numarg >= argc ) || (argv[*numarg] == NULL)){ (void) fprintf( stderr, "%s: missing argument\n",label); exit(EXIT_FAILURE); } if(sscanf(argv[*numarg],"%d",val) != 1){ (void) fprintf(stderr,"Bad integer argument/parameter [%s %s] \n",label,argv[*numarg]); exit(EXIT_FAILURE); } if(*val <= 0){ (void) fprintf(stderr,"Illegal nonpositive integer argument/parameter [%s %s] \n",label,argv[*numarg]); exit(EXIT_FAILURE); } } void setinumeric(int *val,int *numarg,int argc,char **argv,char * label){ (*numarg)++; if( ( *numarg >= argc ) || (argv[*numarg] == NULL)){ (void) fprintf( stderr, "%s: missing argument\n",label); exit(EXIT_FAILURE); } if(sscanf(argv[*numarg],"%d",val) != 1){ (void) fprintf(stderr,"Bad integer argument/parameter [%s %s] \n",label,argv[*numarg]); exit(EXIT_FAILURE); } } void setnonzeronumeric(int *val,int *numarg,int argc,char **argv,char * label){ (*numarg)++; if( ( *numarg >= argc ) || (argv[*numarg] == NULL)){ (void) fprintf( stderr, "%s: missing argument\n",label); exit(EXIT_FAILURE); } if(sscanf(argv[*numarg],"%d",val) != 1){ (void) fprintf(stderr,"Bad integer argument/parameter [%s %s] \n",label,argv[*numarg]); exit(EXIT_FAILURE); } if(*val == 0){ (void) fprintf(stderr,"Illegal zero integer argument/parameter [%s %s] \n",label,argv[*numarg]); exit(EXIT_FAILURE); } } /* floating point formatting */ void ffit(char *string,int ff, int width, int precision, int maxwidth){ int slen,converted; double dtemp; char fstring[32]; /* big enough for any reasonable format string */ converted = sscanf(string,"%lf",&dtemp); if(converted != 1){ /* ugh, not a valid float, best to blow up here*/ (void) fprintf(stderr,"Fatal error: string is not a valid numeric value [%s]\n",string); exit(EXIT_FAILURE); } else { slen=strlen(string); if(width > 0)width +=slen; /*pad, bad idea, but supported */ if(width < 0)width=-width; if(width==0)width=slen; /* it has to fit back into slen, bad idea too, but supported*/ if(precision > width)insane("fatal error: numeric value precision cannot be more than the width "); if(width > maxwidth)insane("fatal error: numeric value width exceeds field width"); if(ff==FFFLOAT){ (void)sprintf(fstring,"%%%d.%df",width,precision); (void)sprintf(string,fstring,dtemp); } else if(ff==FFEXP){ (void)sprintf(fstring,"%%%d.%de",width,precision); (void)sprintf(string,fstring,dtemp); } } } void padit(char *string,int pad,int maxwidth){ int slen,i,tpad; slen = strlen(string); if(pad > 0){ tpad = pad; } else { tpad = -pad; if(slen > tpad){ string[tpad]='\0'; /* truncate it */ return; } tpad -= slen; /* difference it must be padded out */ } if(slen + tpad > maxwidth - 1)insane("extract: fatal error: pad operation overflows buffer"); for(i=slen;i maxwidth - 1)insane("extract: fatal error: slash operation overflows buffer"); /* shift right and pad */ cptr[padnum]='\0'; for(cptr--;cptr>=string;cptr--){ cptr[padnum]=*cptr; if(! (isalnum((unsigned int) *cptr) || *cptr=='_' || *cptr=='.'|| *cptr=='/')){ /* escape this one */ padnum--; cptr[padnum]='\\'; if(padnum==0)break; /* fully slashed out at this point */ } } break; default: insane("extract: fatal programming error: slash not handled in function slashit()"); break; } } /* Modified from miniproc's mpcf_string_justify() */ void justit(char *string,int justify){ int slen,count,lcount; char *from; char *to; switch (justify) { case JUSTNONE: break; case JUSTLEFT: for(from=string; ;from++){ if(*from == '\0')return; /* nothing to justify */ if(*from != ' ')break; } for(to=string; ;to++,from++){ if(*from == '\0')break; *to = *from; } for( ;*to!='\0';to++){*to=' ';} break; case JUSTRIGHT: slen=strlen(string); for(from=&string[slen-1]; ;from--){ if(from < string)return; /* nothing to justify */ if(*from != ' ')break; } for(to=&string[slen-1]; ;to--,from--){ if(from < string)break; *to = *from; } for( ;to >= string;to--){*to=' ';} break; case JUSTCENTER: for(count=0, from=string; ;from++){ if(*from != ' ')break; count++; } lcount=count; slen=strlen(string); for(from=&string[slen-1]; ;from--){ if(from < string || *from != ' ')break; count++; } count = lcount - count/2; /* amount to shift, left if positive */ if(count==0)return; /* nothing to do */ if(count > 0){ /* shift left by this much */ for(to=string,from=&string[count] ; *from!='\0'; from++,to++){ *to=*from; } for( ;*to!='\0'; to++){*to=' ';} } else{ /* shift right by this much */ for(to=&string[slen-1],from=&string[slen-1+count] ; from >= string ; from--,to--){ *to=*from; } for( ; to>=string ; to--){*to=' ';} } break; default: insane("extract: fatal programming error: case not handled in function justit()"); } return; } void trimit(char *string,int trimify){ int slen,trimmed,ws; char *from; char *to; if(trimify == TRIMNONE)return; /* do not trim */ if(*string == '\0')return; /* nothing to trim */ trimmed=0; if(trimify & TRIMRIGHT){ trimmed++; slen=strlen(string); for(from=&string[slen-1]; from>=string ;from--){ if(*from == ' ' || *from == '\t'){ *from='\0'; } else { break; } } } if(trimify & TRIMLEFT){ trimmed++; for(from=string; ;from++){ if(*from == ' ' || *from == '\t')continue; break; } /* from is on the first nonwhitespace character, it might be on a \0 though */ for(to=string; ;to++,from++){ *to = *from; if(*from == '\0')break; } } if(trimify & TRIMCOMPRESS){ /* Replace runs of INTERNAL whitespace with a single space. At this point there are no external whitespaces, these having already been trimmed off. */ trimmed++; ws=0; for(from=to=string; ;from++){ if(*from == ' ' || *from == '\t'){ if(ws==0){ /* on the first char of whitespace write a space, skip the rest */ *to=' '; ws=1; to++; } } else { ws=0; *to=*from; to++; } if(*from=='\0')break; } } if(trimmed==0)insane("extract: fatal programming error: case not handled in function trimit()"); return; } void setstring(char **val,int *numarg,int argc,char **argv,char * label){ (*numarg)++; if( ( *numarg >= argc ) || (argv[*numarg] == NULL)){ (void) fprintf( stderr, "%s: missing argument\n",label); exit(EXIT_FAILURE); } *val = argv[*numarg]; } void setuchar(unsigned char *val,int *numarg,int argc,char **argv,char * label){ char scratch[16]; (*numarg)++; if( ( *numarg >= argc ) || (argv[*numarg] == NULL)){ (void) fprintf( stderr, "%s: missing argument\n",label); exit(EXIT_FAILURE); } if(strlen(argv[*numarg])>15){ (void) fprintf( stderr, "extract: fatal error: %s excessively long argument\n",label); exit(EXIT_FAILURE); } (void) strcpy(scratch,argv[*numarg]); fixupdelimit(scratch); if(scratch[1]!='\0'){ (void) fprintf( stderr, "extract: fatal error: %s argument does not reduce to a single character\n",label); exit(EXIT_FAILURE); } *val=scratch[0]; } void setonechar(char *val,int *numarg,int argc,char **argv,char * label){ (*numarg)++; if( ( *numarg >= argc ) || (argv[*numarg] == NULL)){ (void) fprintf( stderr, "%s: missing argument\n",label); exit(EXIT_FAILURE); } *val = *(argv[*numarg]); } void process_command_line_args(int argc,char **argv, int *maxwidth, int *sc,int *ec, int *sr,int *er, int *rm, int *parse_options, int *all, FIELDNEXUS *gblparams, int *xc, int *dbg, int *tagstate, int *tagtermstate, int *ifn, int *ifonly, int *ifeol, int *ifbol, char **eol, char **bol, char **cols, char **delimit, char **tag, char **tagterm, char **iftermeol, char **iftermbol, char **fileeol, char **filebol, char **indl, int *samelines, int *linenumber, int *merge, char **mdl, int *template, int *handlenull, int *crok){ int numarg=0; int nc,nr; int scset=NO; int ncset=NO; int ecset=NO; int dset=NO; int fromstdin=0; char *delimitv=NULL; char *infstring=NULL; unsigned char hnsubs; int hnset=NO; if(argc <2){ /* no arguments */ emit_help(); } lcl_stdin[0]=stdin; lcl_stdout=stdout; *maxwidth=MYMAXSTRING; *xc=MYMAXRANGES; *rm=ACTEXTRACT; *parse_options=0; /* no options set */ *dbg=NO; *sc=1; gblparams->ttype=MODENONE; /* ==MODECHAR = use characters */ gblparams->pad=0; gblparams->lpad=PADNONE; gblparams->precision=MYDEFPRECISION; gblparams->lprecision=FPNONE; gblparams->ff=FFNONE; gblparams->justify=JUSTNONE; gblparams->trimify=TRIMNONE; gblparams->casify=CASENONE; gblparams->slashify=SLASHNONE; gblparams->ldelim=DELIMNONE; /* ->delimit is defined at the end of this routine */ gblparams->delim = ':'; /* will be replaced if DELIMVAL is specified below */ gblparams->lrs=RSNONE; gblparams->lrcd=RCDNONE; gblparams->lrcs=RCSNONE; gblparams->lrtd=RTDNONE; gblparams->lrts=RTSNONE; gblparams->ersstring=NULL; gblparams->rcd=NULL; gblparams->rcs=NULL; gblparams->rtd=NULL; gblparams->rts=NULL; *ec=TO_END; nc=MYMAXSTRING; *sr=1; *er=0; nr=0; *all=NO; /* do not emit unchanged rows beyond row limits */ **delimit='\0'; *tag=NULL; *tagterm=NULL, *iftermeol=NULL; *iftermbol=NULL; *fileeol=NULL; *filebol=NULL; *tagstate=TAG_NONE; *tagtermstate=TAG_NONE; *ifn=0; *ifonly=NO; *ifeol=NO; *ifbol=NO; *cols=NULL; *indl=NULL; *mdl=NULL; *merge=0; *samelines=NO; /* do not require the same number of lines in each input file */ *linenumber=NO; /* do not emit line numbers before each line */ *eol=malloc(2*sizeof(char)); if(*eol == NULL)insane("extract: fatal error: could not allocate space for default eol string"); (*eol)[0] = '\n'; (*eol)[1] = '\0'; *bol=malloc(sizeof(char)); if(*bol == NULL)insane("extract: fatal error: could not allocate space for default bol string"); (*bol)[0] = '\0'; *template =0; *handlenull=HNULL_RETAIN; *handlenull=YES; while( ++numarg < argc){ if( (lcl_strcasecmp(argv[numarg], "-h")==0) || (lcl_strcasecmp(argv[numarg], "--h")==0) || (lcl_strcasecmp(argv[numarg], "-?")==0) || (lcl_strcasecmp(argv[numarg], "--?")==0) || (lcl_strcasecmp(argv[numarg], "-help")==0) || (lcl_strcasecmp(argv[numarg], "--help")==0) ){ emit_help(); } else if( (lcl_strcasecmp(argv[numarg], "-hcols")==0)){ emit_help_cols(); } else if( (lcl_strcasecmp(argv[numarg], "-hexamples")==0)){ emit_help_examples(); } else if(lcl_strcasecmp(argv[numarg], "-i")==0){ (void)fprintf(stderr,"Version: %s\n",EXVERSTRING); (void)fprintf(stderr,"bugs to: mathog@caltech.edu\n"); (void)fprintf(stderr,"Copyright: 2002,2003,2004,2005 David Mathog and California Institute of Technology\n"); (void)fprintf(stderr,"License terms:\n"); (void)fprintf(stderr," You may run this program on any platform. You may\n"); (void)fprintf(stderr," redistribute the source code of this program subject to\n"); (void)fprintf(stderr," the condition that you do not first modify it in any way.\n"); (void)fprintf(stderr," You may distribute binary versions of this program so long\n"); (void)fprintf(stderr," as they were compiled from unmodified source code. There\n"); (void)fprintf(stderr," is no charge for using this software. You may not charge\n"); (void)fprintf(stderr," others for the use of this software.\n"); exit(EXIT_SUCCESS); } else if(lcl_strcasecmp(argv[numarg], "-wl")==0){ setiposnumeric(maxwidth,&numarg,argc,argv,"-wl"); continue; } else if(lcl_strcasecmp(argv[numarg], "-dbg")==0){ *dbg=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-all")==0){ *all=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-s")==0){ *parse_options |= PARSE_SINGLEDEL; continue; } else if(lcl_strcasecmp(argv[numarg], "-dq")==0){ *parse_options |= PARSE_DBLQUOTES; continue; } else if(lcl_strcasecmp(argv[numarg], "-dqs")==0){ *parse_options |= PARSE_DBLQUOTES; *parse_options |= PARSE_STRIPDQUOTES; continue; } else if(lcl_strcasecmp(argv[numarg], "-in")==0){ /* open as many input files as are present up to MAXINFILE */ infstring=strtok(argv[++numarg],","); for(infiles=0;infiles<=MAXINFILE;infiles++){ if(infstring==NULL)break; if(strcmp(infstring,"-")==0){ if(fromstdin)insane("extract: fatal error: only one input stream may come from stdin"); lcl_stdin[infiles]=stdin; fromstdin=1; } else { lcl_stdin[infiles]=fopen(infstring,"r"); } if(lcl_stdin[infiles]==NULL)insane("extract: fatal error: could not open input file"); infstring=strtok(NULL,","); } if(infiles==0)insane("extract: fatal error: no valid input files"); continue; } else if(lcl_strcasecmp(argv[numarg], "-out")==0){ lcl_stdout=fopen(argv[++numarg],"w"); if(lcl_stdout==NULL)insane("extract: fatal error: could not open output file"); continue; } else if(lcl_strcasecmp(argv[numarg], "-rm")==0){ if(*rm!=ACTEXTRACT)insane("extract: fatal error: conflicting extract/remove/in situ commands"); *rm=ACTREMOVE; continue; } else if(lcl_strcasecmp(argv[numarg], "-is")==0){ if(*rm!=ACTEXTRACT)insane("extract: fatal error: conflicting extract/remove/in situ commands"); *rm=ACTINSITU; continue; } else if(lcl_strcasecmp(argv[numarg], "-crok")==0){ *crok=NO; continue; } else if(lcl_strcasecmp(argv[numarg], "-cl")==0){ if(gblparams->casify != CASENONE)insane("extract: fatal error: conflicting case commands"); gblparams->casify=CASELOWER; continue; } else if(lcl_strcasecmp(argv[numarg], "-cu")==0){ if(gblparams->casify!=CASENONE)insane("extract: fatal error: conflicting case commands"); gblparams->casify=CASEUPPER; continue; } else if(lcl_strcasecmp(argv[numarg], "-cf")==0){ if(gblparams->casify!=CASENONE)insane("extract: fatal error: conflicting case commands"); gblparams->casify=CASEFIRST; continue; } else if(lcl_strcasecmp(argv[numarg], "-bs")==0){ gblparams->slashify=SLASHUNIX; continue; } else if(lcl_strcasecmp(argv[numarg], "-fff")==0){ if(gblparams->ff!=FFNONE)insane("extract: fatal error: conflicting ff commands"); gblparams->ff=FFFLOAT; continue; } else if(lcl_strcasecmp(argv[numarg], "-ffe")==0){ if(gblparams->ff!=FFNONE)insane("extract: fatal error: conflicting ff commands"); gblparams->ff=FFEXP; continue; } else if(lcl_strcasecmp(argv[numarg], "-jl")==0){ if(gblparams->justify!=JUSTNONE)insane("extract: fatal error: conflicting justify commands"); gblparams->justify=JUSTLEFT; continue; } else if(lcl_strcasecmp(argv[numarg], "-jr")==0){ if(gblparams->justify!=JUSTNONE)insane("extract: fatal error: conflicting justify commands"); gblparams->justify=JUSTRIGHT; continue; } else if(lcl_strcasecmp(argv[numarg], "-jc")==0){ if(gblparams->justify!=JUSTNONE)insane("extract: fatal error: conflicting justify commands"); gblparams->justify=JUSTCENTER; continue; } else if(lcl_strcasecmp(argv[numarg], "-trl")==0){ if(gblparams->trimify!=TRIMNONE)insane("extract: fatal error: conflicting trim commands"); gblparams->trimify=TRIMLEFT; continue; } else if(lcl_strcasecmp(argv[numarg], "-trr")==0){ if(gblparams->trimify!=TRIMNONE)insane("extract: fatal error: conflicting trim commands"); gblparams->trimify=TRIMRIGHT; continue; } else if(lcl_strcasecmp(argv[numarg], "-trb")==0){ if(gblparams->trimify!=TRIMNONE)insane("extract: fatal error: conflicting trim commands"); gblparams->trimify=TRIMLEFT | TRIMRIGHT ; continue; } else if(lcl_strcasecmp(argv[numarg], "-trc")==0){ if(gblparams->trimify!=TRIMNONE)insane("extract: fatal error: conflicting trim commands"); gblparams->trimify=TRIMLEFT | TRIMRIGHT | TRIMCOMPRESS; continue; } else if(lcl_strcasecmp(argv[numarg], "-mdl")==0){ setstring(mdl,&numarg,argc,argv,"-mdl"); fixupdelimit(*mdl); continue; } else if(lcl_strcasecmp(argv[numarg], "-indl")==0){ setstring(indl,&numarg,argc,argv,"-indl"); fixupdelimit(*indl); continue; } else if(lcl_strcasecmp(argv[numarg], "-d-")==0){ if(dset==YES)insane("extract: fatal error: conflicting or repeated -d* commands"); dset=YES; gblparams->ldelim=DELIMNONE; continue; } else if(lcl_strcasecmp(argv[numarg], "-dv")==0){ if(dset==YES)insane("extract: fatal error: conflicting or repeated -d* commands"); dset=YES; setstring(&delimitv,&numarg,argc,argv,"-dv"); fixupdelimit(delimitv); gblparams->delim=*delimitv; gblparams->ldelim=DELIMVAL; continue; } else if(lcl_strcasecmp(argv[numarg], "-dt")==0){ if(dset==YES)insane("extract: fatal error: conflicting or repeated -d* commands"); dset=YES; gblparams->ldelim=DELIMTOK; continue; } else if(lcl_strcasecmp(argv[numarg], "-pd")==0){ if(gblparams->lpad != PADNONE)insane("extract: fatal error: conflicting or repeated -pd and/or -fw commands"); setiposnumeric(&(gblparams->pad),&numarg,argc,argv,"-pd"); gblparams->lpad=PADFIELD; continue; } else if(lcl_strcasecmp(argv[numarg], "-fw")==0){ if(gblparams->lpad != PADNONE)insane("extract: fatal error: conflicting or repeated -pd and/or -fw commands"); setiposnumeric(&(gblparams->pad),&numarg,argc,argv,"-fw"); gblparams->pad = -gblparams->pad; gblparams->lpad=PADFIELD; continue; } else if(lcl_strcasecmp(argv[numarg], "-fp")==0){ if(gblparams->lprecision != FPNONE)insane("extract: fatal error: conflicting or repeated -fp commands"); setinumeric(&(gblparams->precision),&numarg,argc,argv,"-fp"); if(gblparams->precision < 0)insane("extract: fatal error: -fp value must be >= 0"); gblparams->lprecision=FPFIELD; continue; } else if(lcl_strcasecmp(argv[numarg], "-n")==0){ *linenumber=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-xc")==0){ setiposnumeric(xc,&numarg,argc,argv,"-xc"); xc += MYMAXCOL; continue; } else if(lcl_strcasecmp(argv[numarg], "-sc")==0){ setinumeric(sc,&numarg,argc,argv,"-sc"); scset=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-ec")==0){ setinumeric(ec,&numarg,argc,argv,"-ec"); ecset=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-nc")==0){ setiposnumeric(&nc,&numarg,argc,argv,"-nc"); ncset=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-sr")==0){ setiposnumeric(sr,&numarg,argc,argv,"-sr"); continue; } else if(lcl_strcasecmp(argv[numarg], "-er")==0){ setiposnumeric(er,&numarg,argc,argv,"-er"); continue; } else if(lcl_strcasecmp(argv[numarg], "-nr")==0){ setiposnumeric(&nr,&numarg,argc,argv,"-nr"); continue; } else if(lcl_strcasecmp(argv[numarg], "-merge")==0){ if(*merge != 0)insane("extract: fatal error: only a single -merge or -unmerge may be used"); setiposnumeric(merge,&numarg,argc,argv,"-merge"); continue; } else if(lcl_strcasecmp(argv[numarg], "-unmerge")==0){ if(*merge != 0)insane("extract: fatal error: only a single -merge or -unmerge may be used"); setiposnumeric(merge,&numarg,argc,argv,"-unmerge"); *merge = -(*merge); continue; } else if(lcl_strcasecmp(argv[numarg], "-template")==0){ setiposnumeric(template,&numarg,argc,argv,"-template"); continue; } else if(lcl_strcasecmp(argv[numarg], "-mt")==0){ gblparams->ttype=MODETOK; if(dset==NO)gblparams->ldelim=DELIMTOK; continue; } else if(lcl_strcasecmp(argv[numarg], "-mc")==0){ gblparams->ttype=MODECHAR; gblparams->ldelim=DELIMNONE; continue; } else if(lcl_strcasecmp(argv[numarg], "-dl")==0){ setstring(delimit,&numarg,argc,argv,"-dl"); fixupdelimit(*delimit); continue; } else if(lcl_strcasecmp(argv[numarg], "-eqlen")==0){ *samelines=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-hnr")==0){ /*default value is HNULL_RETAIN, this line is just here to detect conflicting -hn params */ if(hnset)insane("extract: fatal error: select only one of -hnr, -hnd, -hns, or -hnsubs"); hnset=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-hnd")==0){ if(hnset)insane("extract: fatal error: select only one of -hnr, -hnd, -hns, or -hnsubs"); hnset=YES; *handlenull=HNULL_DELETE; continue; } else if(lcl_strcasecmp(argv[numarg], "-hns")==0){ if(hnset)insane("extract: fatal error: select only one of -hnr, -hnd, -hns, or -hnsubs"); hnset=YES; *handlenull=HNULL_SUBS; continue; } else if(lcl_strcasecmp(argv[numarg], "-hnsubs")==0){ if(hnset)insane("extract: fatal error: select only one of -hnr, -hnd, -hns, or -hnsubs"); hnset=YES; setuchar(&hnsubs,&numarg,argc,argv,"-hnsubs"); if(hnsubs=='\0')insane("extract: fatal error: select -hnsubs may not reduce to the NULL character"); *handlenull=hnsubs; /* force this, so that -hns is not needed with -hnsubs */ continue; } else if(lcl_strcasecmp(argv[numarg], "-if")==0){ setstring(tag,&numarg,argc,argv,"-if"); fixuptag(*tag,tagstate); continue; } else if(lcl_strcasecmp(argv[numarg], "-ifterm")==0){ setstring(tagterm,&numarg,argc,argv,"-ifterm"); fixuptag(*tagterm,tagtermstate); continue; } else if(lcl_strcasecmp(argv[numarg], "-iftermeol")==0){ setstring(iftermeol,&numarg,argc,argv,"-iftermeol"); fixupdelimit(*iftermeol); continue; } else if(lcl_strcasecmp(argv[numarg], "-iftermbol")==0){ setstring(iftermbol,&numarg,argc,argv,"-iftermbol"); fixupdelimit(*iftermbol); continue; } else if(lcl_strcasecmp(argv[numarg], "-fileeol")==0){ setstring(fileeol,&numarg,argc,argv,"-fileeol"); fixupdelimit(*fileeol); continue; } else if(lcl_strcasecmp(argv[numarg], "-filebol")==0){ setstring(filebol,&numarg,argc,argv,"-filebol"); fixupdelimit(*filebol); continue; } else if(lcl_strcasecmp(argv[numarg], "-ifn")==0){ setiposnumeric(ifn,&numarg,argc,argv,"-ifn"); continue; } else if(lcl_strcasecmp(argv[numarg], "-ifonly")==0){ *ifonly=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-ifeol")==0){ *ifeol=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-ifbol")==0){ *ifbol=YES; continue; } else if(lcl_strcasecmp(argv[numarg], "-eol")==0){ setstring(eol,&numarg,argc,argv,"-eol"); fixupdelimit(*eol); continue; } else if(lcl_strcasecmp(argv[numarg], "-bol")==0){ setstring(bol,&numarg,argc,argv,"-bol"); fixupdelimit(*bol); continue; } else if(lcl_strcasecmp(argv[numarg], "-rs")==0){ setstring(&(gblparams->ersstring),&numarg,argc,argv,"-rs"); fixupdelimit(gblparams->ersstring); gblparams->lrs=RSVAL; continue; } else if(lcl_strcasecmp(argv[numarg], "-rcds")==0){ setstring(&(gblparams->rcd),&numarg,argc,argv,"-rcds"); fixupdelimit(gblparams->rcd); gblparams->lrcd=RCDVAL; continue; } else if(lcl_strcasecmp(argv[numarg], "-rcdc")==0){ setstring(&(gblparams->rcd),&numarg,argc,argv,"-rcdc"); fixupdelimit(gblparams->rcd); gblparams->lrcd=RCDVALC; continue; } else if(lcl_strcasecmp(argv[numarg], "-rcss")==0){ setstring(&(gblparams->rcs),&numarg,argc,argv,"-rcss"); fixupdelimit(gblparams->rcs); gblparams->lrcs=RCSVAL; continue; } else if(lcl_strcasecmp(argv[numarg], "-rtds")==0){ setstring(&(gblparams->rtd),&numarg,argc,argv,"-rtds"); fixupdelimit(gblparams->rtd); gblparams->lrtd=RTDVAL; continue; } else if(lcl_strcasecmp(argv[numarg], "-rtdc")==0){ setstring(&(gblparams->rtd),&numarg,argc,argv,"-rtdc"); fixupdelimit(gblparams->rtd); gblparams->lrtd=RTDVALC; continue; } else if(lcl_strcasecmp(argv[numarg], "-rtss")==0){ setstring(&(gblparams->rts),&numarg,argc,argv,"-rtss"); fixupdelimit(gblparams->rts); gblparams->lrts=RTSVAL; continue; } else if(lcl_strcasecmp(argv[numarg], "-cols")==0){ setstring(cols,&numarg,argc,argv,"-cols"); continue; } else { (void) fprintf(stderr,"extract: fatal error: on command line [%s] is not recognized. Use the -h flag to list options.\n",argv[numarg]); exit(EXIT_FAILURE); } } /* sanity checking */ /* cols implies no regular specification of columns */ if(*cols!=NULL){ if( ncset == YES)insane("extract: fatal error: both -cols and -nc present on the command line"); if( ecset == YES)insane("extract: fatal error: both -cols and -ec present on the command line"); if( scset == YES)insane("extract: fatal error: both -cols and -sc present on the command line"); if( *rm != ACTEXTRACT)insane("extract: fatal error: -cols and -is or -rm present on the command line"); } if(ncset == YES){ if(ecset == YES){ insane("extract: fatal error: both -ec and -nc present on the command line"); } else { if(*sc>0){ *ec = *sc + nc - 1; } else { *ec = nc; } } } /* do sanity checking on sc/ec later */ if(nr != 0){ if(*er != 0){ insane("extract: fatal error: both -er and -nr present on the command line"); } else { *er = *sr + nr - 1; } } else{ if(*er != 0 && *er < *sr)insane("extract: fatal error: ending row less than starting row"); } if( *ec > *maxwidth)insane("extract: fatal error: last column specified by -ec, -sc, -nc is beyond -wl limit"); if(gblparams->ttype==MODECHAR && *cols==NULL){ if(*parse_options & PARSE_SINGLEDEL)insane("extract: fatal error: -s specified in character/column mode"); if(*parse_options & PARSE_DBLQUOTES)insane("extract: fatal error: -dq specified in character/column mode"); if(**delimit != '\0')insane("extract: fatal error: -dl specified in character/column mode"); } if(*tag==NULL && *ifn !=0 && *tagterm==NULL)insane("extract: fatal error: -ifn specified without -if"); if(*tag==NULL && *ifonly==YES) insane("extract: fatal error: -ifonly specified without -if"); if(*tag==NULL && *ifeol==YES) insane("extract: fatal error: -ifeol specified without -if"); if(*tag==NULL && *ifbol==YES) insane("extract: fatal error: -ifbol specified without -if"); if(*tag==NULL && *tagterm!=NULL) insane("extract: fatal error: -ifterm specified without -if"); if(*ifn !=0 && *tagterm!=NULL) insane("extract: fatal error: both -ifn and -ifterm specified"); if( (*ifn ==0 && *tagterm==NULL) && *iftermeol!=NULL){ insane("extract: fatal error: -ifn or -ifterm must be specified with -iftermeol"); } if( (*ifn ==0 && *tagterm==NULL) && *iftermbol!=NULL){ insane("extract: fatal error: -ifn or -ifterm must be specified with -iftermbol"); } /* default delimiters are space, column, tab if nothing else was specified */ if(**delimit == '\0')(void)strcpy(*delimit," :\t"); if(gblparams->lrcs == RCSVAL){ if(gblparams->lrcd != RCDVAL && gblparams->lrcd != RCDVALC)insane("extract: fatal error: -rcss specified without -rcds"); if(strlen(gblparams->rcs) != strlen(gblparams->rcd))insane("extract: fatal error: length mismatch: -rcs -rcd"); if(strlen(gblparams->rcs) == 0)insane("extract: fatal error: -rcs may not be an empty string"); } if((gblparams->lrcd == RCDVAL || gblparams->lrcd == RCDVALC) && strlen(gblparams->rcd) == 0)insane("extract: fatal error: -rcd may not be an empty string"); if(gblparams->lrts == RTSVAL){ if(gblparams->lrtd != RTDVAL && gblparams->lrtd != RTDVALC)insane("extract: fatal error: -rtss specified without -rtds"); if(strlen(gblparams->rts) == 0)insane("extract: fatal error: -rtss may not be an empty string"); } if((gblparams->lrtd == RTDVAL ||gblparams->lrtd == RTDVALC) && strlen(gblparams->rtd) == 0)insane("extract: fatal error: -rtds may not be an empty string"); if(*merge < 0){ if(*mdl==NULL)*mdl=*delimit; /* if mdl isn't set explicitly use whatever delimit is */ } if(*template>0){ if(infiles!=2)insane("extract: fatal error: -template requires exactly two input files, the first being the template"); if(*merge>0)insane("extract: fatal error: -template and -merge may not be used together"); if(*template > MYMAXTEMPLATE -1 )insane("extract: fatal error: -template N value is too large"); } } /* convert an lstr to int. Returns 1 if success, 0 if failure. Input string must be decimal and begin with +/-/ or digit. No spaces are allowed before or after. Assumes that overflow can be detected by a sign change. */ int lstrtol(char *string,int length, long *ival){ long sum,lastsum; int i,thesign; thesign=1; for(sum=0,i=0; ittype = MODEPDEFAULT; lclparams->justify = JUSTNONE; lclparams->trimify = TRIMNONE; lclparams->casify = CASENONE; lclparams->slashify = SLASHNONE; lclparams->lpad = PADNONE; lclparams->lprecision = FPNONE; lclparams->ff = FFNONE; lclparams->ldelim = DELIMNONE; lclparams->lrs = RSNONE; lclparams->lrcd = RCDNONE; lclparams->lrcs = RCSNONE; lclparams->lrtd = RTDNONE; lclparams->lrts = RTSNONE; } else if(lstrcasecmp(atoken,length,"+",1) == 0){ if(tcount > 1)insane2("extract, fatal error, -cols contains + out of order ",cmd); lclparams->ttype = MODESDEFAULT; lclparams->justify = JUSTSDEFAULT; lclparams->trimify = TRIMSDEFAULT; lclparams->casify = CASESDEFAULT; lclparams->slashify = SLASHSDEFAULT; lclparams->lpad = PADSDEFAULT; /* default is to use any lpad from command line */ lclparams->pad = 0; lclparams->lprecision = FPSDEFAULT; lclparams->precision = MYDEFPRECISION; lclparams->ff = FFSDEFAULT; lclparams->ldelim = DELIMSDEFAULT; lclparams->delim = ':'; lclparams->lrs = RSSDEFAULT; lclparams->lrcd = RCDSDEFAULT; lclparams->lrcs = RCSSDEFAULT; lclparams->lrtd = RTDSDEFAULT; lclparams->lrts = RTSSDEFAULT; } else if(lstrcasecmp(atoken,length,"p",1) == 0){ if(tcount > 1)insane2("extract, fatal error, -cols contains p out of order ",cmd); lclparams->ttype = MODEPDEFAULT; lclparams->justify = JUSTPDEFAULT; lclparams->trimify = TRIMPDEFAULT; lclparams->casify = CASEPDEFAULT; lclparams->slashify = SLASHPDEFAULT; lclparams->lpad = PADPDEFAULT; /* default is to use any lpad from command line */ lclparams->pad = 0; lclparams->lprecision = FPPDEFAULT; lclparams->precision = MYDEFPRECISION; lclparams->ff = FFPDEFAULT; lclparams->ldelim = DELIMPDEFAULT; lclparams->lrs = RSPDEFAULT; lclparams->lrcd = RCDPDEFAULT; lclparams->lrcs = RCSPDEFAULT; lclparams->lrtd = RTDPDEFAULT; lclparams->lrts = RTSPDEFAULT; } else if(lstrcasecmp(atoken,length,"m-",2) == 0){ lclparams->ttype = MODENONE; } /* same as mc */ else if(lstrcasecmp(atoken,length,"mp",2) == 0){ lclparams->ttype = MODEPDEFAULT; } else if(lstrcasecmp(atoken,length,"m+",2) == 0){ lclparams->ttype = MODESDEFAULT; } else if(lstrcasecmp(atoken,length,"mt",2) == 0){ lclparams->ttype = MODETOK; lclparams->ldelim = DELIMTOK; /* mt implies dt, but it can be toggled later in the [] structure */ } else if(lstrcasecmp(atoken,length,"mc",2) == 0){ lclparams->ttype = MODECHAR; } else if(lstrcasecmp(atoken,length,"c-",2) == 0){ lclparams->casify = CASENONE; } else if(lstrcasecmp(atoken,length,"cp",2) == 0){ lclparams->casify = CASEPDEFAULT; } else if(lstrcasecmp(atoken,length,"c+",2) == 0){ lclparams->casify = CASEPDEFAULT; } else if(lstrcasecmp(atoken,length,"cu",2) == 0){ lclparams->casify = CASEUPPER; } else if(lstrcasecmp(atoken,length,"cl",2) == 0){ lclparams->casify = CASELOWER; } else if(lstrcasecmp(atoken,length,"cf",2) == 0){ lclparams->casify = CASEFIRST; } else if(lstrcasecmp(atoken,length,"b-",2) == 0){ lclparams->slashify = SLASHNONE; } else if(lstrcasecmp(atoken,length,"bp",2) == 0){ lclparams->slashify = SLASHPDEFAULT; } else if(lstrcasecmp(atoken,length,"b+",2) == 0){ lclparams->slashify = SLASHPDEFAULT; } else if(lstrcasecmp(atoken,length,"bs",2) == 0){ lclparams->slashify = SLASHUNIX; } else if(lstrcasecmp(atoken,length,"j-",2) == 0){ lclparams->justify = JUSTNONE; } else if(lstrcasecmp(atoken,length,"jp",2) == 0){ lclparams->justify = JUSTPDEFAULT; } else if(lstrcasecmp(atoken,length,"j+",2) == 0){ lclparams->justify = JUSTSDEFAULT; } else if(lstrcasecmp(atoken,length,"jl",2) == 0){ lclparams->justify = JUSTLEFT; } else if(lstrcasecmp(atoken,length,"jr",2) == 0){ lclparams->justify = JUSTRIGHT; } else if(lstrcasecmp(atoken,length,"jc",2) == 0){ lclparams->justify = JUSTCENTER; } else if(lstrcasecmp(atoken,length,"tr-",3) == 0){ lclparams->trimify = TRIMNONE; } else if(lstrcasecmp(atoken,length,"trp",3) == 0){ lclparams->trimify = TRIMPDEFAULT; } else if(lstrcasecmp(atoken,length,"tr+",3) == 0){ lclparams->trimify = TRIMSDEFAULT; } else if(lstrcasecmp(atoken,length,"trl",3) == 0){ lclparams->trimify = TRIMLEFT; } else if(lstrcasecmp(atoken,length,"trr",3) == 0){ lclparams->trimify = TRIMRIGHT; } else if(lstrcasecmp(atoken,length,"trb",3) == 0){ lclparams->trimify = TRIMLEFT | TRIMRIGHT; } else if(lstrcasecmp(atoken,length,"trc",3) == 0){ lclparams->trimify = TRIMLEFT | TRIMRIGHT | TRIMCOMPRESS; } else if(lstrcasecmp(atoken,length,"ff-",3) == 0){ lclparams->ff = FFNONE; } else if(lstrcasecmp(atoken,length,"ffp",3) == 0){ lclparams->ff = FFPDEFAULT; } else if(lstrcasecmp(atoken,length,"ff+",3) == 0){ lclparams->ff = FFSDEFAULT; } else if(lstrcasecmp(atoken,length,"fff",3) == 0){ lclparams->ff = FFFLOAT; } else if(lstrcasecmp(atoken,length,"ffe",3) == 0){ lclparams->ff = FFEXP; } else if(lstrcasecmp(atoken,length,"d-",2) == 0){ lclparams->ldelim = DELIMNONE; } else if(lstrcasecmp(atoken,length,"dp",2) == 0){ lclparams->ldelim = DELIMPDEFAULT; } else if(lstrcasecmp(atoken,length,"d+",2) == 0){ lclparams->ldelim = DELIMSDEFAULT; } else if(lstrcasecmp(atoken,length,"dt",2) == 0){ lclparams->ldelim = DELIMTOK; } else if(lstrcasecmp(atoken,2,"dv",2) == 0){ if(length-3 > MYMAXDV)insane2("extract: fatal error: invalid -dv value in -cols [] expression = \n",cmd); strncpy(dvtemp,&(atoken[2]),length-2); dvtemp[length-2]='\0'; fixupdelimit(dvtemp); length=strlen(dvtemp); lclparams->ldelim = DELIMVAL; if(length!=1)insane2("extract: fatal error: invalid -dv value in -cols [] expression = \n",cmd); lclparams->delim=*dvtemp; } else if(lstrcasecmp(atoken,length,"rcd-",4) == 0){ lclparams->lrcd = RCDNONE; } else if(lstrcasecmp(atoken,length,"rcdp",4) == 0){ lclparams->lrcd = RCDPDEFAULT; } else if(lstrcasecmp(atoken,length,"rcd+",4) == 0){ lclparams->lrcd = RCDSDEFAULT; } else if(lstrcasecmp(atoken,4,"rcds",4) == 0){ lclparams->lrcd = RCDVAL; lclparams->rcd = malloc(sizeof(char)*(length-2)); if(lclparams->rcd == NULL)insane("extract: fatal error: could not allocate memory"); (void) strncpy(lclparams->rcd,&atoken[4],length-4); lclparams->rcd[length-4]='\0'; fixupdelimit(lclparams->rcd); } else if(lstrcasecmp(atoken,4,"rcdc",4) == 0){ lclparams->lrcd = RCDVALC; lclparams->rcd = malloc(sizeof(char)*(length-2)); if(lclparams->rcd == NULL)insane("extract: fatal error: could not allocate memory"); (void) strncpy(lclparams->rcd,&atoken[4],length-4); lclparams->rcd[length-4]='\0'; fixupdelimit(lclparams->rcd); } else if(lstrcasecmp(atoken,length,"rcs-",4) == 0){ lclparams->lrcs = RCSNONE; } else if(lstrcasecmp(atoken,length,"rcsp",4) == 0){ lclparams->lrcs = RCSPDEFAULT; } else if(lstrcasecmp(atoken,length,"rcs+",4) == 0){ lclparams->lrcs = RCSSDEFAULT; } else if(lstrcasecmp(atoken,4,"rcss",4) == 0){ lclparams->lrcs = RCSVAL; lclparams->rcs = malloc(sizeof(char)*(length-2)); if(lclparams->rcs == NULL)insane("extract: fatal error: could not allocate memory"); (void) strncpy(lclparams->rcs,&atoken[4],length-4); lclparams->rcs[length-4]='\0'; fixupdelimit(lclparams->rcs); } else if(lstrcasecmp(atoken,length,"rtd-",4) == 0){ lclparams->lrtd = RTDNONE; } else if(lstrcasecmp(atoken,length,"rtdp",4) == 0){ lclparams->lrtd = RTDPDEFAULT; } else if(lstrcasecmp(atoken,length,"rtd+",4) == 0){ lclparams->lrtd = RTDSDEFAULT; } else if(lstrcasecmp(atoken,4,"rtds",4) == 0){ lclparams->lrtd = RTDVAL; lclparams->rtd = malloc(sizeof(char)*(length-2)); if(lclparams->rtd == NULL)insane("extract: fatal error: could not allocate memory"); (void) strncpy(lclparams->rtd,&atoken[4],length-4); lclparams->rtd[length-4]='\0'; fixupdelimit(lclparams->rtd); } else if(lstrcasecmp(atoken,4,"rtdc",4) == 0){ lclparams->lrtd = RTDVALC; lclparams->rtd = malloc(sizeof(char)*(length-2)); if(lclparams->rtd == NULL)insane("extract: fatal error: could not allocate memory"); (void) strncpy(lclparams->rtd,&atoken[4],length-4); lclparams->rtd[length-4]='\0'; fixupdelimit(lclparams->rtd); } else if(lstrcasecmp(atoken,length,"rts-",4) == 0){ lclparams->lrts = RTSNONE; } else if(lstrcasecmp(atoken,length,"rtsp",4) == 0){ lclparams->lrts = RTSPDEFAULT; } else if(lstrcasecmp(atoken,length,"rts+",4) == 0){ lclparams->lrts = RTSSDEFAULT; } else if(lstrcasecmp(atoken,4,"rtss",4) == 0){ lclparams->lrts = RTSVAL; lclparams->rts = malloc(sizeof(char)*(length-2)); if(lclparams->rts == NULL)insane("extract: fatal error: could not allocate memory"); (void) strncpy(lclparams->rts,&atoken[4],length-4); lclparams->rts[length-4]='\0'; fixupdelimit(lclparams->rts); } else if(lstrcasecmp(atoken,length,"rs-",3) == 0){ lclparams->lrs = RSNONE; } else if(lstrcasecmp(atoken,length,"rsp",3) == 0){ lclparams->lrs = RSPDEFAULT; } else if(lstrcasecmp(atoken,length,"rs+",3) == 0){ lclparams->lrs = RSSDEFAULT; } else if(lstrcasecmp(atoken,3,"rss",3) == 0){ lclparams->lrs = RSVAL; lclparams->ersstring = malloc(sizeof(char)*(length-2)); if(lclparams->ersstring == NULL)insane("extract: fatal error: could not allocate memory"); (void) strncpy(lclparams->ersstring,&atoken[3],length-3); lclparams->ersstring[length-3]='\0'; fixupdelimit(lclparams->ersstring); } else if(lstrcasecmp(atoken,length,"pd-",3) == 0){ lclparams->lpad = PADNONE; } else if(lstrcasecmp(atoken,length,"pdp",3) == 0){ lclparams->lpad = PADPDEFAULT; } else if(lstrcasecmp(atoken,length,"pd+",3) == 0){ lclparams->lpad = PADSDEFAULT; } else if(lstrcasecmp(atoken,2,"pd",2) == 0){ if(! lstrtol(&atoken[2],length-2,(long *) &(lclparams->pad)) || (lclparams->precision < 0)){ insane2("extract: fatal error: invalid -pd value in -cols [] expression = \n",cmd); } lclparams->lpad=PADFIELD; } else if(lstrcasecmp(atoken,length,"fw-",3) == 0){ lclparams->lpad = PADNONE; } else if(lstrcasecmp(atoken,length,"fwp",3) == 0){ lclparams->lpad = PADPDEFAULT; } else if(lstrcasecmp(atoken,length,"fw+",3) == 0){ lclparams->lpad = PADSDEFAULT; } else if(lstrcasecmp(atoken,2,"fw",2) == 0){ if(! lstrtol(&atoken[2],length-2,(long *) &(lclparams->pad)) || (lclparams->pad < 0)){ insane2("extract: fatal error: invalid -fw value in -cols [] expression = \n",cmd); } lclparams->lpad=PADFIELD; lclparams->pad = -(lclparams->pad); } else if(lstrcasecmp(atoken,length,"fp-",3) == 0){ lclparams->lprecision = FPNONE; } else if(lstrcasecmp(atoken,length,"fpp",3) == 0){ lclparams->lprecision = FPPDEFAULT; } else if(lstrcasecmp(atoken,length,"fp+",3) == 0){ lclparams->lprecision = FPSDEFAULT; } else if(lstrcasecmp(atoken,2,"fp",2) == 0){ if(! lstrtol(&atoken[2],length-2,(long *) &(lclparams->precision)) || (lclparams->precision < 0)){ insane2("extract: fatal error: invalid -fp value in -cols [] expression = \n",cmd); } lclparams->lprecision=FPFIELD; } else { insane2("extract: fatal error: invalid -cols [] expression = ",cmd); } } else { /* not :, must be the range */ get_range(atoken,length,sc,nc,inc); } /* test for : or range */ if(tcount == 0){ atoken=strtok_variant(cmd,":",PARSE_ESCAPES,"",&thedelim,&start,&length); } else { atoken=strtok_variant(NULL,":",PARSE_ESCAPES,"",&thedelim,&start,&length); } tcount++; } } void get_range(char *string, int length, int *sc, int *nc, int *inc){ int comma,comma2,ok; *inc=1; *sc=1; ok=0; for(comma=0 ; comma= length){ /* [#] */ if(lstrtol(string,length,(long *)sc) && *sc != 0){ *nc=*sc; ok=1; } } else if(comma == 0){ /* [,#],[,,#] or [,#,#] */ if(comma2 >= length - 1){ /* [,#] or [,#,] */ if(lstrtol(&string[comma+1],length-comma-1,(long *)nc) && *nc != 0)ok=1; } else { /* [,,#] or [,#,#] */ if(comma2==1){ /* [,,#] */ *nc=TO_END; if(lstrtol(&string[comma2+1],length-comma2-1,(long *)inc) && *inc != 0)ok=1; } else { /* [,#,#] */ if(lstrtol(&string[comma+1],comma2-comma-1,(long *)nc) && *nc != 0 && lstrtol(&string[comma2+1],length-comma2-1,(long *)inc) && *inc != 0)ok=1; } } } else if(comma ==length -1){ /* [#,] */ if(lstrtol(string,comma,(long *)sc) && *sc != 0){ *nc=TO_END; ok=1; } } else{ /* [#,#], [#,#,], [#,#,#], [#,,], [#,,#] */ if(comma2 >= length-1){ /* [#,#,], [#,#], [#,,]*/ if(comma2>comma+1){ /* [#,#,], [#,#]*/ if(lstrtol(string,comma,(long *)sc) && *sc != 0 && lstrtol(&string[comma+1],comma2-comma-1,(long *)nc) && *nc != 0)ok=1; } else{ /* [#,,] */ *nc=TO_END; if(lstrtol(string,comma,(long *)sc) && *sc != 0)ok=1; } } else { /* [#,#,#] [#,,#] */ if(comma2>comma+1){ /* [#,#,#]*/ if(lstrtol(string,comma,(long *)sc) && *sc != 0 && lstrtol(&string[comma+1],comma2-comma-1,(long *)nc) && *nc != 0 && lstrtol(&string[comma2+1],length-comma2-1,(long *)inc) && *inc != 0 )ok=1; } else { /* [#,,#] */ *nc=TO_END; if(lstrtol(string,comma,(long *)sc) && *sc != 0 && lstrtol(&string[comma2+1],length-comma2-1,(long *)inc) && *inc != 0 )ok=1; } } } if(!ok)insane2("extract: fatal error: invalid range in [] field: ",string); } /* add_op processes [] or regular string. where a [] is processed like this: t = token c = character (defaults to whatever -tok set on command line) + = process (pad, etc) - = noprocess (defaults to process) # = just the one column ,# = first through this column #, = this column through last #,# = column range */ void add_op(char *cmd, int start, int length, FIELDNEXUS *fieldlist, STRINGNEXUS *isslist, int *dynamic, int *fieldcount, int *maketokens, char *buffer, FIELDNEXUS *gblparams){ int sc,nc,inc; char holdc; int newlength; FIELDNEXUS lclparams; if(length==0)return; if(cmd[start]=='['){ /* column field processing */ if(length==2)insane("extract: fatal error: -cols contains an empty [] "); start++; /* trim off the [] */ length -= 2; cmd[start+length]='\0'; classify_op(&cmd[start],&sc,&nc,&inc,&lclparams); /* static addressing of character ranges goes in isslist, dynamic (token) addressing is handled at run time */ if(lclparams.ttype==MODESDEFAULT){ fieldlist[*fieldcount].ttype = gblparams->ttype; } else if(lclparams.ttype==MODEPDEFAULT){ fieldlist[*fieldcount].ttype = MODENONE; /* same as MODECHAR */ } else { fieldlist[*fieldcount].ttype = lclparams.ttype; } if(fieldlist[*fieldcount].ttype==MODETOK){ fieldlist[*fieldcount].iss = sc; /* first token in the buffer */ fieldlist[*fieldcount].issend = nc; /* last token to process */ fieldlist[*fieldcount].issinc = inc; /* increment by */ *maketokens=YES; /* at least one token required, so make all */ } else { isslist[*dynamic].string = buffer; isslist[*dynamic].start = sc; /* first character in the buffer */ isslist[*dynamic].send = nc; /* last character */ isslist[*dynamic].check = YES; /* need to check for line ends at emission */ isslist[*dynamic].delimit = 0; /* delimiter always 0 for character strings */ (*dynamic)++; fieldlist[*fieldcount].iss = *dynamic; /* where the string will live */ fieldlist[*fieldcount].issend = *dynamic; /* only one string to process start==end */ fieldlist[*fieldcount].issinc = 1; /* irrelevant, but set to something */ } /* merge the global (command line) settings with the field settings so that when it comes time to emit only the field settings need to be checked. S fields match settings, P match program defaults, and anything else uses the values returned in lclparams */ if (lclparams.lpad == PADSDEFAULT ){ fieldlist[*fieldcount].lpad = gblparams->lpad; fieldlist[*fieldcount].pad = gblparams->pad; } else if (lclparams.lpad == PADPDEFAULT ){ fieldlist[*fieldcount].lpad = PADNONE; fieldlist[*fieldcount].pad = 0; } else { fieldlist[*fieldcount].lpad = lclparams.lpad; fieldlist[*fieldcount].pad = lclparams.pad; } if (lclparams.lprecision == FPSDEFAULT ){ fieldlist[*fieldcount].lprecision = gblparams->lprecision; fieldlist[*fieldcount].precision = gblparams->precision; } else if (lclparams.lprecision == FPPDEFAULT ){ fieldlist[*fieldcount].lprecision = FPNONE; fieldlist[*fieldcount].precision = MYDEFPRECISION; } else { fieldlist[*fieldcount].lprecision = lclparams.lprecision; fieldlist[*fieldcount].precision = lclparams.precision; } if (lclparams.ldelim == DELIMSDEFAULT ){ fieldlist[*fieldcount].ldelim = gblparams->ldelim; fieldlist[*fieldcount].delim = gblparams->delim; } else if (lclparams.ldelim == DELIMPDEFAULT ){ fieldlist[*fieldcount].ldelim = DELIMTOK; fieldlist[*fieldcount].delim = ':'; /* will be overwritten when token is made */ } else { /* DELIMNONE DELIMTOK or DELIMVAL */ fieldlist[*fieldcount].ldelim = lclparams.ldelim; fieldlist[*fieldcount].delim = lclparams.delim; /* from a dlN construct */ } if (lclparams.justify == JUSTSDEFAULT){ fieldlist[*fieldcount].justify = gblparams->justify; } else if (lclparams.justify == JUSTPDEFAULT){ fieldlist[*fieldcount].justify = JUSTNONE; } else { fieldlist[*fieldcount].justify = lclparams.justify; } if (lclparams.trimify == TRIMSDEFAULT){ fieldlist[*fieldcount].trimify = gblparams->trimify; } else if (lclparams.trimify == TRIMPDEFAULT){ fieldlist[*fieldcount].trimify = TRIMNONE; } else { fieldlist[*fieldcount].trimify = lclparams.trimify; } if (lclparams.casify == CASESDEFAULT){ fieldlist[*fieldcount].casify = gblparams->casify; } else if (lclparams.casify == CASEPDEFAULT){ fieldlist[*fieldcount].casify = CASENONE; } else { fieldlist[*fieldcount].casify = lclparams.casify; } if (lclparams.ff == FFSDEFAULT){ fieldlist[*fieldcount].ff = gblparams->ff; } else if (lclparams.ff == FFPDEFAULT){ fieldlist[*fieldcount].ff = FFNONE; } else { fieldlist[*fieldcount].ff = lclparams.ff; } if (lclparams.slashify == SLASHSDEFAULT){ fieldlist[*fieldcount].slashify = gblparams->slashify; } else if (lclparams.slashify == SLASHPDEFAULT){ fieldlist[*fieldcount].slashify = SLASHNONE; } else { fieldlist[*fieldcount].slashify = lclparams.slashify; } if (lclparams.lrs == RSSDEFAULT){ fieldlist[*fieldcount].lrs = gblparams->lrs; fieldlist[*fieldcount].ersstring = gblparams->ersstring; } else if (lclparams.lrs == RSPDEFAULT || lclparams.lrs == RSNONE){ fieldlist[*fieldcount].lrs = RSNONE; fieldlist[*fieldcount].ersstring = NULL; } else { fieldlist[*fieldcount].lrs = RSVAL; fieldlist[*fieldcount].ersstring = lclparams.ersstring; } if (lclparams.lrcd == RCDSDEFAULT){ fieldlist[*fieldcount].lrcd = gblparams->lrcd; fieldlist[*fieldcount].rcd = gblparams->rcd; } else if (lclparams.lrcd == RCDPDEFAULT || lclparams.lrcd == RCDNONE){ fieldlist[*fieldcount].lrcd = RCDNONE; fieldlist[*fieldcount].rcd = NULL; } else { fieldlist[*fieldcount].lrcd = RCDVAL; fieldlist[*fieldcount].rcd = lclparams.rcd; } if (lclparams.lrcs == RCSSDEFAULT){ fieldlist[*fieldcount].lrcs = gblparams->lrcs; fieldlist[*fieldcount].rcs = gblparams->rcs; } else if (lclparams.lrcs == RCSPDEFAULT || lclparams.lrcs == RCSNONE){ fieldlist[*fieldcount].lrcs = RCSNONE; fieldlist[*fieldcount].rcs = NULL; } else { fieldlist[*fieldcount].lrcs = RCSVAL; fieldlist[*fieldcount].rcs = lclparams.rcs; } if (lclparams.lrtd == RTDSDEFAULT){ fieldlist[*fieldcount].lrtd = gblparams->lrtd; fieldlist[*fieldcount].rtd = gblparams->rtd; } else if (lclparams.lrtd == RTDPDEFAULT || lclparams.lrtd == RTDNONE){ fieldlist[*fieldcount].lrtd = RTDNONE; fieldlist[*fieldcount].rtd = NULL; } else { fieldlist[*fieldcount].lrtd = RTDVAL; fieldlist[*fieldcount].rtd = lclparams.rtd; } if (lclparams.lrts == RTSSDEFAULT){ fieldlist[*fieldcount].lrts = gblparams->lrts; fieldlist[*fieldcount].rts = gblparams->rts; } else if (lclparams.lrts == RTSPDEFAULT || lclparams.lrts == RTSNONE){ fieldlist[*fieldcount].lrts = RTSNONE; fieldlist[*fieldcount].rts = NULL; } else { fieldlist[*fieldcount].lrts = RTSVAL; fieldlist[*fieldcount].rts = lclparams.rts; } } else { /* static string processing - simplest, little processing required */ holdc=cmd[start+length]; cmd[start+length]='\0'; fixupdelimit(&(cmd[start])); /* handle \n \t \t \123 \[ \]*/ newlength=strlen(&(cmd[start])); cmd[start+length]=holdc; isslist[*dynamic].string = cmd; /* includes offset already as safe here */ isslist[*dynamic].delimit = 0; /* delimiter always 0 for commands */ isslist[*dynamic].start = start+1; /* start of this string */ isslist[*dynamic].send = start+newlength; /* end of the string */ isslist[*dynamic].check = NO; /* no need to check for line ends at emission */ (*dynamic)++; fieldlist[*fieldcount].iss = *dynamic; /* one string to process only */ fieldlist[*fieldcount].issend = *dynamic; fieldlist[*fieldcount].issinc = 1; /* default to 1*/ fieldlist[*fieldcount].ttype = MODECHAR; /* equivalent to MODECHAR */ fieldlist[*fieldcount].pad = 0; /* irrelevant, but set to something */ fieldlist[*fieldcount].lpad = PADNONE; /* no pad */ fieldlist[*fieldcount].precision = 0; /* irrelevant, but set to something */ fieldlist[*fieldcount].lprecision= FPNONE; /* no precision */ fieldlist[*fieldcount].ff = FFNONE; /* not a floating point */ fieldlist[*fieldcount].ldelim = DELIMNONE;/* not a token, so no delim */ fieldlist[*fieldcount].delim = '\0'; /* irrelevant, but set to something */ fieldlist[*fieldcount].justify = JUSTNONE; /* no justify */ fieldlist[*fieldcount].trimify = TRIMNONE; /* no trim */ fieldlist[*fieldcount].casify = CASENONE; /* no case change */ fieldlist[*fieldcount].slashify = SLASHNONE; /* no case change */ fieldlist[*fieldcount].lrs = RSNONE; /* no replace string */ fieldlist[*fieldcount].ersstring = NULL; /* irrelevant, but set to something */ fieldlist[*fieldcount].lrcd = RCDNONE; /* no replace character delete string */ fieldlist[*fieldcount].rcd = NULL; /* irrelevant, but set to something */ fieldlist[*fieldcount].lrcs = RCSNONE; /* no replace character substitute string */ fieldlist[*fieldcount].rcs = NULL; /* irrelevant, but set to something */ fieldlist[*fieldcount].lrtd = RTDNONE; /* no replace text delete string */ fieldlist[*fieldcount].rtd = NULL; /* irrelevant, but set to something */ fieldlist[*fieldcount].lrts = RTSNONE; /* no replace text substitute string */ fieldlist[*fieldcount].rts = NULL; /* irrelevant, but set to something */ } (*fieldcount)++; } /* realends() converts from relative [lstart,lend] format to C offset (starting at 0) rstart,rend format. This routine traps only one kind of error - start or lend = 0*/ void realends(int *rstart, int *rend, int lstart, int lend, int length){ int relative; if(lend == 0 || lstart == 0){ (void) fprintf(stderr,"extract, fatal error, invalid range start %d, end %d, length %d\n", lstart,lend,length); exit(EXIT_FAILURE); } relative=0; if(length > 0){ if(lend>0){ /* absolute column, from front */ *rend = lend - 1; /* 1->0, now in C offset format */ } else if(lend <0){ /* relative column from end */ *rend = length + lend; /* 1->0, now in C offset format */ relative=1; } if(lstart>0){ /* absolute column, from front */ *rstart = lstart - 1; /* 1->0, now in C offset format */ } else if(lstart < 0){ /* relative column, from end */ *rstart = length + lstart; /* 1->0, now in C offset format */ relative=1; } if(relative){ /* [3,-1] should not be an error even on short lines, but [3,2] should be */ if(*rstart > *rend)*rstart=*rend=2000000000; } } else { if(lend>0 && lstart>0){ /* absolute column, both ends */ *rend = lend - 1; /* 1->0, now in C offset format */ *rstart = lstart - 1; /* 1->0, now in C offset format */ } else if(lend <0 && lstart >0){ /* relative column from end, absolute start */ *rend=*rstart = lstart - 1; } else if(lend >0 && lstart <0){ /* relative column from start, absolute end */ *rend=*rstart = lend - 1; } else{ /* relative column both ends, set it to something out of range */ *rend=*rstart = 2000000000; } } } /* debugging routine, not normally incorporated into code */ void dumpcommands(FIELDNEXUS *fieldlist,STRINGNEXUS *isslist, int dynamic, int tcount, int fieldcount, int linelen, char *cols){ int i,j,rstart,rend,r2end; for(i=0;itcount - 1 (MODETOK) 0<->dynamic - 1 (MODECHAR) Use badj to trap these cases. The first is sometimes allowed but the second one is always an error. */ if(jend < jstart){ /* ie, [mt:30,-30] on a 3 token line goes to start = 29, end =-27 */ (void) fprintf(stderr,"extract, fatal error, range start %d, end %d\n",jstart+1,jend+1); exit(EXIT_FAILURE); } j= (fsinc>0 ? jstart : jend); /* start the pointer at the right end */ for(lasttoken=NO;;j += fsinc){ *piece='\0'; /* be sure to preinitialize the piece string */ if(fsinc>0){ if(j>jend)break; if(j==jend || j+fsinc>jend )lasttoken=YES; } else{ if(j tcount - 1) && (ttype == MODETOK) )badj=YES; if( (j >= dynamic ) && (ttype != MODETOK) )badj=YES; if( badj==YES ){ if( ttype == MODETOK ){ /* ie 3 input tokens on [-10,3] or [1,5] */ if((lrs == RSVAL) && (ersstring != NULL) )(void)strcpy(piece,ersstring); /* use the replacement string instead */ } else{ /* any other mode this is a programming error */ insane("extract, fatal programming error, message 1001"); } } else { if(ttype == MODETOK){ if(j >= tcount)break; /* no more tokens on line for this field */ jssout = j + dynamic; } else { jssout = j; } /* when going [1,5,1] use delimiter on each token, but when going 1,5,-1 use delimiter on token before. Otherwise if there are just 5 tokens the first one out will use the default token since it has none of its own*/ if(fsinc >0){ idelim =isslist[jssout].delimit; } else { if(jssout-1>=0){ idelim =isslist[jssout-1].delimit; } else { idelim ='\0'; } } if(isslist[jssout].string==NULL){ /* empty token */ piece[0]='\0'; if((lrs == RSVAL) && (ersstring!=NULL))(void)strcpy(piece,ersstring); } else { /* valid token */ isstart=isslist[jssout].start; isend =isslist[jssout].send; if(isslist[jssout].string == cols) { r2end = MYMAXSTRING; } else { r2end = linelen; } realends(&rstart,&rend,isstart,isend,r2end); if(rstart>rend)insane("extract: fatal error: column range invalid"); /* CAREFUL! At this point rstart/rend can be way left or way right of the valid data at 0<->linelength - 1. First lock start at no less than 0 and then test on rend to see what, if anything, can be emitted. */ if(rstart < 0)rstart=0; if(rstart >= r2end || rend < 0){ /* no regular field to emit */ if((lrs == RSVAL) && (ersstring!=NULL))(void)strcpy(piece,ersstring); } else { /* there is something to emit */ if(isslist[jssout].check == YES){ /* may need to truncate the line */ if(rend >= r2end)rend = r2end-1; } rlen = rend - rstart + 1; (void) strncpy(piece,&(isslist[jssout].string[ rstart ]),rlen); piece[rlen]='\0'; /* terminate the string, may be a zero length string!!! */ } } /* valid token */ } /* badj test */ /* something exists to output */ if(frcd != RCDNONE) rcit(piece,rcd,rcs,frcd); if(frtd != RTDNONE) rtit(piece,maxwidth,rtd,rts,frtd); if(fslashify != SLASHNONE) slashit(piece,fslashify,maxwidth); if(fff != FFNONE){ ffit(piece,fff,fpad,fprecision,maxwidth); } else { if(fpad != PADNONE) padit(piece,fpad,maxwidth); } if(ftrimify != TRIMNONE) trimit(piece,ftrimify); if(fjustify != JUSTNONE) justit(piece,fjustify); if(fcasify != CASENONE) caseit(piece,fcasify); if(!bolout){ if(!ifbol)(void)fprintf(lcl_stdout,"%s",bol); bolout=1; } if(linenumber==YES){ (void)fprintf(lcl_stdout,"%8d:",linecount); /* emit the line number */ linenumber=NO; } (void)fprintf(lcl_stdout,"%s",piece); /* emit the processed (maybe) field */ /* When multiple tokens are emitted the default is to emit their delimiters if they are (potentially) followed by another token from the same or another []. If they are followed by a static string or the end of the line then do not emit the delimiter. */ dodelim=YES; if(dtype == DELIMNONE){ dodelim=NO; } else if(lasttoken){ /* end of a run of delimiters, so maybe special */ if(i+1>=fieldcount){ /* followed by end of line - do not emit delimiter */ dodelim=NO; } else if(i+1 < fieldcount && fieldlist[i+1].ttype==MODECHAR){ dodelim=NO; /* followed by a nontoken field - do not emit delimiter */ } } if( dodelim == YES){ if(dtype == DELIMTOK){ if(idelim != '\0'){ (void)fprintf(lcl_stdout,"%c",idelim); } else{ (void)fprintf(lcl_stdout,"%c",vdelim); } } else if(dtype == DELIMVAL){ (void)fprintf(lcl_stdout,"%c",vdelim); } } } /* all tokens processed (or single string for column mode */ } /* all fields processed */ if(!ifeol)(void)fprintf(lcl_stdout,"%s",eol); } void regular_to_cols(char **cols, char *buffer,int sc,int ec, int rm){ if(rm==ACTEXTRACT){ (void)sprintf(buffer,"[%d,%d]",sc,ec); } if(rm==ACTREMOVE){ if(sc > 1 || sc < 1){ /* sc is a real column measured from the front */ if (ec == TO_END || ec == -1){ /* these are the same thing!!! */ (void)sprintf(buffer,"[-:m+:d+:1,%d]",sc-1); } else if(ec != 0) { (void)sprintf(buffer,"[-:m+:d+:1,%d][-:m+:d+:%d,]",sc-1,ec+1); } } else{ if (ec != TO_END || ec == -1){ (void)sprintf(buffer,"[-:m+:d+:%d,]",ec+1); } else { (void)sprintf(buffer,"[]"); } } } if(rm==ACTINSITU){ if(sc > 1 || sc < 1){ if (ec != TO_END){ (void)sprintf(buffer,"[-:m+:d+:1,%d][%d,%d][-:m+:d+:%d,]",sc-1,sc,ec,ec+1); } else { (void)sprintf(buffer,"[-:m+:d+:1,%d][%d,%d]",sc-1,sc,ec); } } else{ if (ec != TO_END){ (void)sprintf(buffer,"[%d,%d][-:m+:d+:%d,]",sc,ec,ec+1); } else { (void)sprintf(buffer,"[%d,%d]",sc,ec); } } } *cols=malloc(sizeof(char) * (strlen(buffer) + 1)); /* for terminator */ if(*cols == NULL){ insane("extract: fatal error: could not allocate cols buffer"); } (void) strcpy(*cols,buffer); } /* Thank you Fred! ** Designation: StriStr ** ** Call syntax: char *stristr(char *String, char *Pattern) ** ** Description: This function is an ANSI version of strstr() with ** case insensitivity. ** ** Return item: char *pointer if Pattern is found in String, else ** pointer to 0 ** ** Rev History: 02/03/94 Fred Cole ** ** Hereby donated to public domain. */ #include #include #include char *stristr(char *String, char *Pattern) { char *pptr, *sptr, *start; unsigned int slen, plen; for (start = String, pptr = Pattern, slen = strlen(String), plen = strlen(Pattern); /* while string length not shorter than pattern length */ slen >= plen; start++, slen--) { /* find start of pattern in string */ while (toupper(*start) != toupper(*Pattern)) { start++; slen--; /* if pattern longer than string */ if (slen < plen) return(NULL); } sptr = start; pptr = Pattern; while (toupper(*sptr) == toupper(*pptr)) { sptr++; pptr++; /* if end of pattern then pattern was found */ if ('\0' == *pptr) return (start); } } return(NULL); } int tagcheck(char *buffer, char *tag, int tagstate, int taglength){ int retval; int buflen; int ttagstate; /* default values */ retval=0; if(tag==NULL)return 1; /* no IF has been defined */ if(taglength==0){ if(buffer==NULL || *buffer=='\0')retval=1; } else{ ttagstate=tagstate & TAG_MASK; if(ttagstate==TAG_NONE){ retval=1; } else if(ttagstate==TAG_ANYWHERE){ if(stristr(buffer,tag)!=NULL)retval=1; } else if(ttagstate==TAG_FRONT){ if(0==lstrcasecmp(tag, taglength, buffer, taglength)){ retval=1; } } else if(ttagstate==TAG_END){ buflen=strlen(buffer); if(0==lstrcasecmp(tag, taglength, &(buffer[buflen-taglength]), taglength)){ retval=1; } } else if(ttagstate==(TAG_END | TAG_FRONT)){ /*tag_front and and tag_end may be used together */ buflen=strlen(buffer); if( (0==lstrcasecmp(tag, taglength, buffer, taglength)) && (0==lstrcasecmp(tag, taglength, &(buffer[buflen-taglength]), taglength)) ){ retval=1;} } else{ insane("extract: fatal programming error: tagcheck"); } } if(tagstate & TAG_NOT){ retval = retval ^ 1; /* flip retval */ } return retval; } /* get next line from one or more files. If any of them terminate, all must terminate. Also checks on some error conditions for the presence of embedded nulls and distinguishes between that and a line that is too long in the error message. However, both conditions are fatal unless null suppression is turned on, in which case input lines with NULLs are converted as soon as they are read. This is not the default state because it slows things down. Note: the terminal character on a normal fgets read on any OS is supposed to be \n regardless of whether or not the OS encodes it as LF, CRLF, or something else. However since it is common to move files from windows to unix, and those files read on linux really then did end in \r\n, the routine below was considering the first \r or \n as the end of the line. However that caused problems when there were embedded single CR or LF in the file. Consequently, the current version now checks the last character to see if it is \n and then converts it to a \0. NO OTHER PROCESSING TAKES PLACE! If it is desired to remove \r on files transferred from a Windows -> Unix system use the rcds command. */ int getnextline(char *buffer,int maxwidth,FILE **lcl_stdin,int infiles, char *indl, int samelines, int template, int handlenull,int crok){ char *cptr; int i; int terminated; int firstchar; int remwidth; int status; static char *hbuffer=NULL; static int dirty=0; size_t cterm; unsigned int sfg_status; int hasnull; if(template > 0){ if(hbuffer==NULL){ hbuffer=malloc(sizeof(char)*(maxwidth+1)); if(hbuffer==NULL)insane("extract: fatal error: could not allocate memory for template operation"); } status = 1; /* Note that for the purpose of this template method ONLY a cterm=0 is sufficient to indicate an EOF. If the file happens to end with: text without an that's ok, and cterm will not then be zero. Also neither the template file nor the file it is compared to may contain empty lines. */ /* ALWAYS get a line from the template. super_fgets does not leave a \n the end */ remwidth=maxwidth; sfg_status = super_fgets(buffer, remwidth, lcl_stdin[0], &cterm, handlenull); if(crok && (sfg_status & SFG_CRLF)){ /* silently nip off the \r character if present on a CRLF */ cterm--; buffer[cterm]='\0'; } if(cterm == 0){ if(dirty)insane("extract: fatal error: input file contained a record not present in the template file"); status=0; /* no more templateable (word???) lines */ } else if(sfg_status & SFG_READERROR){ insane("extract: fatal error: fatal read error from template file."); } else if(sfg_status & SFG_EMBEDDED_NULL && handlenull < 0){ insane("extract: fatal error: found embedded null characters in template file. Use -hnd, -hns, or -hnsubs"); } else { if(!dirty){ /* must load data into hbuffer */ sfg_status = super_fgets(hbuffer, remwidth, lcl_stdin[1], &cterm, handlenull); if(crok && (sfg_status & SFG_CRLF)){ /* silently nip off the \r character if present on a CRLF */ cterm--; hbuffer[cterm]='\0'; dirty=1; } if( cterm == 0){ /* empty line no more in input, remainder from template, leave dirty=0*/ if(samelines==YES)insane2("extract: fatal error: present in template, missing in file: ",buffer); (void) strcat(buffer,indl); } else if(sfg_status & SFG_EMBEDDED_NULL && handlenull < 0){ insane("extract: fatal error: found embedded null characters in file. Use -hnd, -hns, or -hnsubs"); } else if(sfg_status & SFG_READERROR){ insane("extract: fatal error: fatal read error from file."); } else { dirty=1; } } if(dirty){ if(strncmp(buffer,hbuffer,template)==0){ /* this is a match, return the one from hbuffer*/ (void) strcpy(buffer,hbuffer); dirty=0; /* force read next time */ } else { /* no match */ if(samelines==YES)insane2("extract: fatal error: present in template, missing in file: ",buffer); (void) strcat(buffer,indl); } } } return status; } hasnull=0; terminated=0; firstchar=0; remwidth=maxwidth; for(i=0;i0 && indl != NULL){ /* put on the line delimiter BEFORE the next line except on the FIRST input file and the LAST input file. Do this even if the preceding input file had no input line */ for(cptr=indl;*cptr!='\0';){ buffer[firstchar]=*cptr; cptr++; firstchar++; if(firstchar >= maxwidth){ insane("extract: fatal error: sum of input lines was wider than -wl"); } } } if(lcl_stdin[i]==NULL){ /* This input file already terminated */ terminated++; } else{ sfg_status = super_fgets(&buffer[firstchar], remwidth, lcl_stdin[i], &cterm, handlenull); if(sfg_status != 0){ if(sfg_status & SFG_READERROR){ insane("extract: fatal error: fatal read error during processing"); } if(sfg_status & SFG_BUFFER_FULL){ insane("extract: fatal error: sum of input lines was wider than -wl"); } if(sfg_status & SFG_EMBEDDED_NULL && handlenull < 0){ insane("extract: fatal error: found embedded null characters. Use -hnd, -hns, or -hnsubs"); } if(sfg_status & SFG_EOF){ lcl_stdin[i]=NULL; terminated++; } if(crok && (sfg_status & SFG_CRLF)){ /* silently nip off the \r character if present on a CRLF */ cterm--; buffer[firstchar+cterm]='\0'; } } } remwidth = maxwidth - cterm; firstchar = firstchar + cterm; } if((samelines==YES) && (terminated != 0) && (terminated != infiles)) insane("extract: fatal error: input files are different lengths"); if(terminated == infiles){ return 0; /* everything has been read */ } return 1; } /* create a partial buffer from the first token plus N tokens more. If first is NULL it is calculated from the input string. Returns the number of tokens found this pass. Sets *first to NULL if no more data can be extracted from the line. */ int nextpartialbuffer(char *pbuffer, char **first, int *flen, char **string, int parse_options, int ntokens, char *mdl){ int start,tlen; /* 1 start variables and a length variable*/ char delimit; char *atoken; char *sstring; char *hstring; int alen,i; if(*string==NULL){ *first=NULL; /* will need a new first token */ return 0; /* will need a new input buffer */ } else if(**string=='\0'){ *first=NULL; /* will need a new first token */ pbuffer[0]='\0'; return 0; } hstring=sstring=*string; if(*first==NULL){ *first=strtok_variant(sstring,mdl,parse_options,"",&delimit,&start,&tlen); if(first==NULL){ /* empty line */ *pbuffer='\0'; return 0; } sstring=NULL; (void) strncpy(pbuffer,*first,tlen); *flen=alen=tlen; } else { (void) strncpy(pbuffer,*first,*flen); alen=*flen; } pbuffer[alen]='\0'; for(i=1;i<=ntokens;i++){ /* NOTICE, i starts from 1 here!!! */ atoken=strtok_variant(sstring,mdl,parse_options,"",&delimit,&start,&tlen); sstring=NULL; if(atoken==NULL){ /* No more tokens left */ *first=NULL; /* will need a new first token */ return 0; } else { pbuffer[alen]=*mdl; alen++; strncpy(&(pbuffer[alen]),atoken,tlen); alen += tlen; pbuffer[alen]='\0'; (*string) = &(hstring[start+tlen+1]); if(**string == '\0'){ *first=NULL; /* will need a new first token */ return i; } } } return i; } /* super_fgets is a wrapper around getc. It does the following: A: (like fgets) reads from a stream B: (like fgets) accepts a preallocated buffer C: (like fgets) accepts the size of that preallocated buffer D: (unlike fgets) terminates the characters read with a '\0' in all cases. E: (unlike fgets) returns the position of the terminating null = number of characters in the buffer (size_t). F: (unlike fgets) sets a status integer where the bits are as defined in the table far above (SFG_*). Status of 0 is a normal read. G: (unlike fgets) either ignores, deletes, or replaces embedded NULL characters depending on the value of enchar: <0: leave the embedded null characters in the string 0: delete the embedded null characters >0: replace the embedded null characters with enchar Limitations: Not a drop in fgets() replacement! Cannot process arbitrary binary data from a pipe on Windows and probably some other platforms. It will usually do so if the data is read from a file opened "rb" - but in that case the EOL characters will show up explicitly here. */ int super_fgets(char *string, size_t size, FILE *stream, size_t *cterm, int enchar){ size_t lastslot; /* the last character cell in the buffer */ size_t icterm; /* internal cterm value */ int status; /* status value */ int readthis; /* the character which was read */ icterm = 0; status = 0; lastslot = size-1; while(1){ /* Buffer full? Signal this and exit the read loop. */ if(icterm == lastslot){ status |= SFG_BUFFER_FULL; break; } /* get the next character from the input stream */ readthis=getc(stream); /* EOF or read error? Figure out which and exit the read loop. */ if(readthis == EOF){ if(feof(stream)){ status |= SFG_EOF; } else { status |= SFG_READERROR; } break; } /* Line terminator? Signal EOL. Return characters read so far, but NOT the terminator. Check for and signal \r. If found, leave the \r. Exit the read loop. */ if(readthis == '\n'){ status |= SFG_EOL; if( (icterm > 0) && (string[icterm-1]=='\r')) status |= SFG_CRLF; break; } /* Normal characters or Embedded null characters? Normal: Store as is. Embedded Null: These are utterly toxic to most C string routines. Handle them here as instructed by enchar. Signal an embedded null was read from the input.*/ if(readthis != '\0'){ string[icterm] = readthis; } else { status |= SFG_EMBEDDED_NULL; /* warn about embedded null characters */ if (enchar == 0) { continue; } /* delete null char */ else if(enchar > 0){ string[icterm] = enchar; } /* replace null char */ else { string[icterm] = '\0'; } /* store null char */ /* enchar < 0 instructs that it be stored as a regular character */ } icterm++; } string[icterm]='\0'; /* terminate the string!!!! */ *cterm = icterm; /* location of the terminator = length of string without terminator */ return status; } int main(int argc, char *argv[]){ /* command line values */ int maxwidth; int sc,ec; int sr,er; int all; int parse_options; int rm; char cdelimit[MYSHORTSTRING]; char *delimit=&cdelimit[0]; char *tag; char *tagterm; char *iftermeol; char *iftermbol; char *fileeol; char *filebol; char *indl; char *mdl; int merge; int forcebuffer; int samelines; int template; int linenumber; int tagstate,taglength; /* if */ int tagtermstate,tagtermlength; /* ifterm */ int ifn,ifncount,ifonly,ifeol,ifbol; int emitteol=0; char *eol; char *bol; char *buffer=NULL; /* buffer to hold lines */ char *mbuffer=NULL; /* buffer to hold 2nd line if merge is going on */ char *piece; /* buffer to hold the current piece */ char *ftoken=NULL; /* pointer to first token in unmerge operations */ char *obuffer; int xc; /* max ranges, used for -cols mode */ int linecount; int mlinecount; int dbg; int linelen=0; int mlinelen=0; int flen; /* first token length */ int mdllen; int dynamic; /* where token pointers start in isslist */ int fieldcount; /* number of ranges to process */ int maketokens; /* logical, if NO then no need to tokenize input */ int tcount; int firstline=YES; /* first line special when -merge is active */ int usembuf; /* special case, when a merge terminates use mbuffer instead of buffer ONCE */ char *cols; char *mtoken; int *olinecount; int handlenull; int crok; FIELDNEXUS *fieldlist; FIELDNEXUS gblparams; STRINGNEXUS *isslist; lcl_stdout=NULL; lcl_stdin[0]=NULL; tcount=0; ifncount=0; taglength=0; tagtermlength=0; process_command_line_args(argc,argv, &maxwidth,&sc,&ec,&sr,&er,&rm,&parse_options, &all,&gblparams,&xc,&dbg,&tagstate,&tagtermstate,&ifn,&ifonly, &ifeol,&ifbol,&eol,&bol,&cols,&delimit,&tag,&tagterm, &iftermeol,&iftermbol, &fileeol,&filebol, &indl,&samelines, &linenumber,&merge,&mdl,&template,&handlenull,&crok); buffer=malloc(sizeof(char)*(maxwidth+1)); if(buffer == NULL){ insane("extract: fatal error: could not allocate character buffer"); } if(merge!=0){ mbuffer=malloc(sizeof(char)*(maxwidth+1)); if(mbuffer == NULL){ insane("extract: fatal error: could not allocate merge buffer"); } obuffer=mbuffer; olinecount=&mlinecount; } else{ obuffer=buffer; olinecount=&linecount; } piece=malloc(sizeof(char)*(maxwidth+1)); if(piece == NULL){ insane("extract: fatal error: could not allocate secondary character buffer"); } fieldlist=malloc(xc*sizeof(FIELDNEXUS)); if(fieldlist == NULL){ insane("extract: fatal error: could not allocate field list buffer"); } isslist=malloc(xc*sizeof(STRINGNEXUS)); if(isslist == NULL){ insane("extract: fatal error: could not allocate string list buffer"); } if(tagstate!=TAG_NONE)taglength=strlen(tag); if(tagtermstate!=TAG_NONE)tagtermlength=strlen(tagterm); mdllen=0; if(mdl!=NULL)mdllen=strlen(mdl); if(cols == NULL){ /* regular command line processing */ regular_to_cols(&cols,buffer,sc,ec,rm); } init_process_type(cols,buffer,fieldlist,isslist,&dynamic,&fieldcount,&maketokens, &gblparams); if(filebol!=NULL){ (void) fprintf(lcl_stdout,"%s",filebol); } linecount=0; usembuf=NO; forcebuffer=NO; mtoken=NULL; while( 1 ){ if(merge>0){ if(usembuf == YES){ usembuf=NO; strcpy(buffer,mbuffer); linelen=mlinelen; mlinecount=linecount; if(er>0 && linecount>=er){ forcebuffer=YES; linecount--; } } if(!getnextline(mbuffer,maxwidth,lcl_stdin,infiles,indl,samelines,template,handlenull,crok)){ if(er>0 && linecount>er)break; forcebuffer=YES; /* buffer still holds data, force it out */ } else { mlinelen=strlen(mbuffer); linecount++; } } else if(merge < 0){ if(! nextpartialbuffer(buffer,&ftoken,&flen,&mtoken,parse_options,-merge,mdl)){ if(!getnextline(mbuffer,maxwidth,lcl_stdin,infiles,indl,samelines,template,handlenull,crok))break; linecount++; mlinecount=linecount; mlinelen=strlen(mbuffer); mtoken=mbuffer; (void) nextpartialbuffer(buffer,&ftoken,&flen,&mtoken,parse_options,-merge,mdl); } linelen=strlen(buffer); } else { if(!getnextline(buffer,maxwidth,lcl_stdin,infiles,indl,samelines,template,handlenull,crok))break; linelen=strlen(buffer); linecount++; } if(ifncount>0)ifncount--; if(linecount < sr){ if(all){ (void) fprintf(lcl_stdout,"%s",bol); if(linenumber==YES)(void) fprintf(lcl_stdout,"%8d:",linecount); (void) fprintf(lcl_stdout,"%s%s",obuffer,eol); } } else if(er != 0 && linecount > er){ if(all){ (void) fprintf(lcl_stdout,"%s",bol); if(linenumber==YES)(void) fprintf(lcl_stdout,"%8d:",linecount); (void) fprintf(lcl_stdout,"%s%s",obuffer,eol); } else { exit(EXIT_SUCCESS); } } else { /* in row range as seen by linecount */ if(merge>0){ /* merge in as many lines as are required up to -er */ if(forcebuffer==NO){ if(firstline==YES){ firstline=NO; (void) strcpy(buffer,mbuffer); linelen=mlinelen; mlinecount=linecount; continue; /* go back for more input lines */ } else { if(0==lcl_strcmprange(mbuffer,buffer,0,merge-1,NO)){ if(mlinelen + linelen + mdllen -merge >= maxwidth)insane("extract: fatal error: input buffer overflow while merging lines"); if(mdllen>0){ (void) strncpy(&(buffer[linelen]),mdl,mdllen); linelen += mdllen; } if(mlinelen>merge){ (void) strncpy(&(buffer[linelen]),&(mbuffer[merge]),mlinelen-merge); linelen += (mlinelen - merge); } if(er==0 || linecount0 || ifncount == IFFOREVER ){ if(maketokens==YES)makealltokens(buffer,delimit,isslist,parse_options, "", dynamic, &tcount); if(dbg==YES)dumpcommands(fieldlist,isslist,dynamic,tcount,fieldcount,linelen,cols); emit_output(piece,fieldlist,isslist,tcount,dynamic,fieldcount,maxwidth,linelen, cols, ifeol, ifbol, eol, bol, linenumber, *olinecount); } else { if(ifonly==NO){ (void) fprintf(lcl_stdout,"%s",bol); if(linenumber==YES)(void) fprintf(lcl_stdout,"%8d:",*olinecount); (void) fprintf(lcl_stdout,"%s%s",obuffer,eol); } } /* emit the if block termination string? */ if( emitteol){ if(iftermeol!=NULL){ (void) fprintf(lcl_stdout,"%s",iftermeol); } emitteol=0; } } /* linecount conditions */ if(forcebuffer==YES)break; } if( ((ifncount == IFFOREVER) || (ifncount >0)) ){ if(iftermbol!=NULL)(void) fprintf(lcl_stdout,"%s",iftermbol); if(iftermeol!=NULL)(void) fprintf(lcl_stdout,"%s",iftermeol); } if(linecount < sr)insane("extract: fatal error: start row is beyond the end of the input data"); if(fileeol!=NULL){ (void) fprintf(lcl_stdout,"%s",fileeol); } exit(EXIT_SUCCESS); }