Pickwiki - User contributions [en]

NotepadPlusPlus

2021-10-07T20:43:34Z

Rkgozar: /* Qmbasic = */

<nowiki>Notepad++</nowiki> can be configured to do syntax highlighting for BASIC programs.

== Setup ==

# Change directory to <tt><nowiki>%APPDATA%\Notepad++</nowiki></tt>
# <nowiki>Copy/paste</nowiki> the xml from the appropriate section below into your <tt><nowiki>userDefineLang.xml</nowiki></tt>
# (Optional) You can add/change/remove keywords in the xml file, or use <nowiki>notepad++'s</nowiki> User Define dialog to change them (and their respective colors)

== Qmbasic ==
Following is a version for QM basic. Click Language > qmbasic to enable syntax highlighting.

<pre>
<NotepadPlus>
<UserLang name="qmbasic" ext="qmb">
<Settings>
<Global caseIgnored="yes" />
<TreatAsSymbol comment="no" commentLine="no" />
<Prefix words1="no" words2="no" words3="no" words4="no" />
</Settings>
<KeywordLists>
<Keywords name="Delimiters">"'0"'0</Keywords>
<Keywords name="Folder+"></Keywords>
<Keywords name="Folder-"></Keywords>
<Keywords name="Operators">' - ! " # % & ( ) , . / : ; ? @ [ \ ] ^ ` { | } ~ + < = ></Keywords>
<Keywords name="Comment">1 1 2 2 0! 0* 0REM 0!* 0*!</Keywords>
<Keywords name="Words1">$CATALOG $CATALOGUE $DEBUG $DEFINE $EXECUTE $IFDEF $IFNDEF $ELSE $ENDIF $INCLUDE $INSERT $LIST $MODE $NO.CATALOGUE $NO.XREF $NOCASE.STRINGS $PAGE $QMCALL $STOP #CATALOG #CATALOGUE #DEBUG #DEFINE #EXECUTE #IFDEF #IFNDEF #ELSE #ENDIF #INCLUDE #INSERT #LIST #MODE #NO.CATALOGUE #NO.XREF #NOCASE.STRINGS #PAGE #QMCALL #STOP</Keywords>
<Keywords name="Words2">ABORT ABORTE ABORTM BEGIN BREAK CALL CASE CHAIN CLASS CLEAR CLEARCOMMON CLEARDATA CLEARFILE CLEARINPUT CLEARSELECT CLOSE CLOSE.SOCKET CLOSESEQ COMMIT COMMON CONTINUE CONVERT CREATE CREATE.FILE CRT DATA DEBUG DEFFUN DEL DELETE DELETELIST DELETESEQ DELETEU DIM DIMENSION DISINHERIT DISPLAY DPARSE DPARSE.CSV ECHO ELSE END ENTER EQU EQUATE ERRMSG EXECUTE EXIT FILE FILELOCK FILEUNLOCK FIND FINDSTR FLUSH FLUSH.DH.CACHE FOOTING FOR FORMLIST FORMLISTV FUNCTION GET GETLIST GO GOSUB GOTO HEADING HUSH IF IN INHERIT INPUT INPUT INPUTCLEAR INPUTCSV INPUTERR INPUTFIELD INS KEYEDIT KEYEXIT KEYTRAP LOCAL LOCATE LOCK LOGMSG LOOP MARK.MAPPING MAT MATBUILD MATPARSE MATREAD MATREADCSV MATREADL MATREADU MATWRITE MATWRITEU NAP NOBUF NULL OFF ON OPEN OPENPATH OPENSEQ OS.EXECUTE OSDELETE OSREAD OSWRITE PAGE PAUSE PERFORM PRECISION PRINT PRINTCSV PRINTER PRINTERR PRIVATE PROCREAD PROCWRITE PROGRAM PROMPT PUBLIC PUT RANDOMIZE READ READBLK READCSV READL READLIST READNEXT READSEQ READU READV READVL READVU RECORDLOCKL RECORDLOCKU RELEASE REMARK REMOVE REMOVE.BREAK.HANDLER REPEAT RESTORE.SCREEN RETURN ROLLBACK RQM SAVELIST SEEK SELECT SELECTE SELECTINDEX SELECTLEFT SELECTN SELECTRIGHT SELECTV SET.ARG SET.BREAK.HANDLER SET.EXIT.STATUS SET.TIMEZONE SETLEFT SETPU SETREM SETRIGHT SLEEP SSELECT START STATUS STOP STOPE STOPM SUBROUTINE TCLREAD THEN TIMEOUT TO TRANSACTION TTYSET UNLOCK UNTIL VOID WAKE WEOFSEQ WHILE WRITE WRITEBLK WRITECSV WRITESEQ WRITESEQF WRITEU WRITEV WRITEVU</Keywords>
<Keywords name="Words3">ABS ABSS ACCEPT.SOCKET.CONNECTION ACOS ALPHA ANDS ARG ARG.COUNT ARG.PRESENT ASCII ASIN ASSIGNED ATAN BINDKEY BITAND BITNOT BITOR BITRESET BITSET BITTEST BITXOR CATALOGUED CATS CHANGE CHAR CHECKSUM CHILD COL1 COL2 COMPARE COMPARES CONFIG CONNECT.PORT CONVERT COS COUNT COUNTS CREATE.SERVER.SOCKET CROP CSVDQ DATE DCOUNT DECRYPT DELETE DFPART DIR DIV DOWNCASE DQUOTE DQUOTES DTX EBCDIC ENCRYPT ENV EPOCH EQS EXP EXTRACT FCONTROL FIELD FIELDS FIELDSTORE FILE.EVENT FILEINFO FMT FMTS FOLD FOLDS GES GETARG.) GET.MESSAGES GET.PORT.PARAMS GETNLS GETPU GETREM GTS ICONV ICONVS IDIV IFS INDEX INDEXS INDICES INMAT INSERT INT ITYPE KEYCODE KEYIN KEYINC KEYINR KEYREADY LEN LENS LES LISTINDEX LN LOCATE LOWER LTS MATCHES MATCHESS MATCHFIELD MAX MAXIMUM MD5 MIN MINIMUM MOD MODS MVDATE MVDATE.TIME MVEPOCH MVTIME NEG NEGS NES NOT NOTS NUM NUMS OBJECT OBJINFO OCONV OCONVS OPEN.SOCKET ORS OS.ERROR OUTERJOIN PRINTER.SETTING PWR QUOTE QUOTES RAISE RDIV READ.SOCKET RECORDLOCKED REM REMOVE REMOVEF REPLACE REUSE RND ROUNDDOWN ROUNDUP RTRANS SAVE.SCREEN SELECTINFO SENTENCE SEQ SERVER.ADDR SET.PORT.PARAMS SET.SOCKET.MODE SETNLS SHIFT SIN SOCKET.INFO SOUNDEX SOUNDEXS SPACE SPACES SPLICE SQRT SQUOTE SQUOTES STATUS STR STRS SUBR SUBSTITUTE SUBSTRINGS SUM SUMMATION SWAP SWAPCASE SYSTEM TAN TERMINFO TESTLOCK TIME TIMEDATE TRANS TRIM TRIMB TRIMBS TRIMF TRIMFS TRIMS TTYGET UNASSIGNED UPCASE VOCPATH VSLICE WAIT.FILE.EVENT WRITE.SOCKET XLATE XTD</Keywords>
<Keywords name="Words4">@ABORT.CODE @ABORT.MESSAGE @ACCOUNT @AM @ANS @COMMAND @COMMAND.STACK @CONV @CRTHIGH @CRTWIDE @DATA @DATA.PENDING @DATE @DAY @DICTRECS @DS @FALSE @FILE.NAME @FILENAME @FM @FMT @GID @HOSTNAME @ID @IM @IP.ADDR @ITYPE.MODE @LEVEL @LOGNAME @LPTRHIGH @LPTRWIDE @MONTH @NB @NI @OPTION @PARASENTENCE @PATH @PIB @POB @QMSYS @RECORD @SELECTED @SENTENCE @SEQNO @SIB @SM @SOB @STDFIL @SVM @SYS.BELL @SYSTEM.RETURN.CODE @TERM.TYPE @TIME @TM @TRANSACTION.ID @TRANSACTION.LEVEL @TRIGGER.RETURN.CODE @TRUE @TTY @UID @USER @USER.NO @USER.RETURN.CODE @USER0 @USER1 @USER2 @USER3 @USER4 @USERNO @VM @VOC @WHO @YEAR @YEAR4</Keywords>
</KeywordLists>
<Styles>
<WordsStyle name="DEFAULT" styleID="11" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="FOLDEROPEN" styleID="12" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="FOLDERCLOSE" styleID="13" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD1" styleID="5" fgColor="FF00FF" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD2" styleID="6" fgColor="FF0000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD3" styleID="7" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD4" styleID="8" fgColor="800000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="COMMENT" styleID="1" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="COMMENT LINE" styleID="2" fgColor="00BF00" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="NUMBER" styleID="4" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="OPERATOR" styleID="10" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER1" styleID="14" fgColor="804000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER2" styleID="15" fgColor="804000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER3" styleID="16" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
</Styles>
</UserLang>
</NotepadPlus></pre>

== Universe ==
Following is a version for Universe basic. Click Language > basic to enable syntax highlighting.

<pre>
<NotepadPlus>
<UserLang name="basic" ext="">
<Settings>
<Global caseIgnored="yes" />
<TreatAsSymbol comment="no" commentLine="yes" />
<Prefix words1="no" words2="no" words3="no" words4="no" />
</Settings>
<KeywordLists>
<Keywords name="Delimiters">"'0"'0</Keywords>
<Keywords name="Folder+"></Keywords>
<Keywords name="Folder-"></Keywords>
<Keywords name="Operators">' - ! " # ( ) , / : ; [ ] ^ + < = ></Keywords>
<Keywords name="Comment">1 1 2 2 0* 0!</Keywords>
<Keywords name="Words1">$CHAIN $COPYRIGHT $DEFINE $EJECT $IFDEF $IFNDEF $INCLUDE $INSERT $MAP $OPTIONS $PAGE $UNDEFINE ABORT ABORTM ACTIVATEKEY AND AUTHORIZATION AUXMAP BEFORE BEGIN CASE TRANSACTION BREAK BSCAN CLEAR CLEARDATA CLEARFILE CLEARPROMPTS CLEARSELECT CLOSE CLOSESEQ COMMIT COMMON CONTINUE CONVERT CREATE CRT DATA DEACTIVATEKEY DEBUG DEFFUN DEL DELETE DELETELIST DIMENSION DIM DISABLEDEC DISPLAY DO ECHO ENABLEDEC END EOF ELSE EQUATE EQU ERRMSG EXIT FILELOCK FILEUNLOCK FIND FINDSTR FLUSH FOOTING FOR FORMLIST FROM FUNCTION GET GETX GETLIST GOSUB GOTO GO GROUPSTORE HEADING HUSH IF INCLUDE INPUT INPUTCLEAR INPUTDISP INPUTDP INPUTERR INPUTIF INPUTNULL INPUTTRAP INS ISOLATION KEYEDIT KEYEXIT KEYTRAP LET LIT LOCATE LOCK LOCKED LOOP MAT MATBUILD MATCH MATCHES MATPARSE MATREAD MATREADL MATREADU MATWRITE MATWRITEU NAP NEXT NOBUF NULL ON OPEN OPENCHECK OPENDEV OPENSEQ OR PAGE PRECISION PRINT PRINTER PRINTERR PROCREAD PROCWRITE PROGRAM PROMPT RANDOMIZE READ READBLK READL READLIST READNEXT READSEQ READT READU READV READVL READVU RECORDLOCK RELEASE REMOVE REPEAT RETURN REVREMOVE REWIND ROLLBACK SEEK SELECT SELECTE SELECTINDEX SEND SETREM SETTING SLEEP SSELECT STOP STOPM STORAGE SUBROUTINE SWAP TABSTOP THEN TIMEOUT TO TPRINT TTYCTL TTYGET TTYSET UNLOCK UPRINT UNTIL WAITING WEOF WEOFSEQ WHILE WRITE WRITEBLK WRITELIST WRITESEQ WRITESEQF WRITET WRITEU WRITEV WRITEVU LT LE EQ GE GT NE</Keywords>
<Keywords name="Words2">$FALSE $TRUE CALL CAPTURING CHAIN ENTER EXECUTE PERFORM RETURNING</Keywords>
<Keywords name="Words3">ABS ABSS acceptConnection ACOS addAuthenticationRule addCertificate addRequestParameter ADDS ALPHA amInitialize amReceiveMsg amReceiveRequest amSendMsg amSendRequest amSendResponse amTerminate analyzeCertificate ANDS ASCII ASIN ASSIGNED ATAN BITAND BITNOT BITOR BITRESET BITSET BITTEST BITXOR BYTE BYTELEN BYTETYPE BYTEVAL CATS CENTURY.PIVOT CHANGE CHAR CHARS CHECKSUM closeSocket CloseXMLData COL1 COL2 COMPARE COS COSH COUNT COUNTS createCertification createCertRequest createRequest createSecureRequest createSecurityContext DATE DBTOXML DCOUNT DESCRINFO DIGEST DIV DIVS DOWNCASE DQUOTE DTX EBCDIC ENCODE ENCRYPT EQS EREPLACE EXCHANGE EXP EXTRACT FADD FDIV FFIX FFLT FIELD FIELDS FIELDSTORE FILEINFO FIX FMT FMTDP FMTS FMTSDP FMUL FOLD FOLDDP FSUB generateKey GES getCipherSuite getSocketErrorMessage getSocketErrorMessage getSocketOptions getHTTPDefault GETLOCALE GETREM getSocketInformation GROUP GTS ICHECK ICONV ICONVS IFS ILPROMPT INDEX INDEXS INDICES initSecureServerSocket initSecureSocket INMAT INSERT INT ISNULL ISNULLS ITYPE KEYIN LEFT LEN LENDP LENS LENSDP LN loadSecurityContext LOCALEINFO LOWER LTS MATCHFIELD MAXIMUM MINIMUM MOD MODS MULS NEG NEGS NES NOT NOTS NUM NUMS OCONV OCONVS openSecureSocket openSocket OpenXMLData ORS PrePareXML protocolLogging PWR QUOTE RAISE readSocket ReadXMLData REAL RECORDLOCKED ReleaseXML REM REPLACE REUSE RIGHT RND RPC.CALL RPC.CONNECT RPC.DISCONNECT saveSecurityContext SADD SCMP SDIV SELECTINFO SENTENCE SEQ SEQS setAuthenticationDepth setCipherSuite setClientAuthentication setPrivateKey setRandomSeed setHTTPDefault setRequestHeader SETLOCALE setSocketOptions showSecurityContext SIGNATURE SIN SINH SMUL SOAPCreateRequest SOAPGetDefault SOAPGetFault SOAPGetResponseHeader SOAPSetRequestBody SOAPSetRequestContent SOAPSetRequestHeader SOAPRequestWrite SOAPSetDefault SOAPSetParameters SOAPSubmitRequest SOUNDEX SPACE SPACES SPLICE SQRT SQUOTE SSUB STATUS STR STRS submitRequest SUBR SUBS SUBSTRS SUM SUMMATION SYSTEM TAN TANH TERMINFO TIME TIMEDATE TPARM TRANS TRIM TRIMB TRIMBS TRIMF TRIMFS TRIMS UNASSIGNED UNICHAR UNISEQ UNISEQS UPCASE XLATE XTD</Keywords>
<Keywords name="Words4">@ABORT.CODE @ACCOUNT @AM @ANS @AUTHORIZARTION @COMMAND @COMMAND.STACK @CONV @CRTHIGH @CRTWIDE @DATA.PENDING @DATE @DAY @DICT @FALSE @FILE.NAME @FILENAME @FM @FORMAT @HDBC @HEADER @HENV @HSTMT @ID @IM @ISOLATION @LEVEL @LOGNAME @LPTRHIGH @LPTRWIDE @MONTH @MV @NB @ND @NEW @NI @NS @NULL @NULL.STR @NV @OLD @OPTION @PARASETENCE @PATH @RECOUNT @RECORD @RECUR0 @RECUR1 @RECUR2 @RECUR3 @RECUR4 @SCHEMA @SELECTED @SM @SQL.CODE @SQL.DATE @SQL.ERROR @SQL.STATE @SQL.TIME @SQL.WARNING @SQLPROC.NAME @SQLPROC.TSLEVEL @STDFIL @SVM @SYS.BELL @SYSTEM.RETURN.CODE @SYSTEM.SET @TERM.TYPE @TIME @TM @TRANSACTION @TRANSACTION.LEVEL @TRUE @TTY @USER0 @USER1 @USER2 @USER3 @USER4 @USERNO @USER.NO @USER.RETURN.CODE @VM @WHO @YEAR @YEAR4</Keywords>
</KeywordLists>
<Styles>
<WordsStyle name="DEFAULT" styleID="11" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" fontSize="10" />
<WordsStyle name="FOLDEROPEN" styleID="12" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="1" />
<WordsStyle name="FOLDERCLOSE" styleID="13" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="1" />
<WordsStyle name="KEYWORD1" styleID="5" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD2" styleID="6" fgColor="FF0080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD3" styleID="7" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="0" fontSize="10" />
<WordsStyle name="KEYWORD4" styleID="8" fgColor="FF0000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="COMMENT" styleID="1" fgColor="008040" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="COMMENT LINE" styleID="2" fgColor="008000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="NUMBER" styleID="4" fgColor="FF0000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="OPERATOR" styleID="10" fgColor="000080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER1" styleID="14" fgColor="800080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER2" styleID="15" fgColor="800080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER3" styleID="16" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
</Styles>
</UserLang>
</NotepadPlus>
</pre>

NotepadPlusPlus

2021-10-07T20:42:02Z

Rkgozar: /* Universe */

<nowiki>Notepad++</nowiki> can be configured to do syntax highlighting for BASIC programs.

== Setup ==

# Change directory to <tt><nowiki>%APPDATA%\Notepad++</nowiki></tt>
# <nowiki>Copy/paste</nowiki> the xml from the appropriate section below into your <tt><nowiki>userDefineLang.xml</nowiki></tt>
# (Optional) You can add/change/remove keywords in the xml file, or use <nowiki>notepad++'s</nowiki> User Define dialog to change them (and their respective colors)

== Qmbasic ===
Following is a version for QM basic. Click Language > qmbasic to enable syntax highlighting.

<pre>
<NotepadPlus>
<[[UserLang]] name="qmbasic" ext="qmb">
<Settings>
<Global caseIgnored="yes" />
<[[TreatAsSymbol]] comment="no" commentLine="no" />
<Prefix words1="no" words2="no" words3="no" words4="no" />
</Settings>
<KeywordLists>
<Keywords name="Delimiters">"'0"'0</Keywords>
<Keywords name="Folder+"></Keywords>
<Keywords name="Folder-"></Keywords>
<Keywords name="Operators">' - ! " # % & ( ) , . / : ; ? @ [ \ ] ^ ` { | } ~ + < = ></Keywords>
<Keywords name="Comment">1 1 2 2 0! 0* 0REM 0!* 0*!</Keywords>
<Keywords name="Words1">$CATALOG $CATALOGUE $DEBUG $DEFINE $EXECUTE $IFDEF $IFNDEF $ELSE $ENDIF $INCLUDE $INSERT $LIST $MODE $NO.CATALOGUE $NO.XREF $NOCASE.STRINGS $PAGE $QMCALL $STOP #CATALOG #CATALOGUE #DEBUG #DEFINE #EXECUTE #IFDEF #IFNDEF #ELSE #ENDIF #INCLUDE #INSERT #LIST #MODE #NO.CATALOGUE #NO.XREF #NOCASE.STRINGS #PAGE #QMCALL #STOP</Keywords>
<Keywords name="Words2">ABORT ABORTE ABORTM BEGIN BREAK CALL CASE CHAIN CLASS CLEAR CLEARCOMMON CLEARDATA CLEARFILE CLEARINPUT CLEARSELECT CLOSE CLOSE.SOCKET CLOSESEQ COMMIT COMMON CONTINUE CONVERT CREATE CREATE.FILE CRT DATA DEBUG DEFFUN DEL DELETE DELETELIST DELETESEQ DELETEU DIM DIMENSION DISINHERIT DISPLAY DPARSE DPARSE.CSV ECHO ELSE END ENTER EQU EQUATE ERRMSG EXECUTE EXIT FILE FILELOCK FILEUNLOCK FIND FINDSTR FLUSH FLUSH.DH.CACHE FOOTING FOR FORMLIST FORMLISTV FUNCTION GET GETLIST GO GOSUB GOTO HEADING HUSH IF IN INHERIT INPUT INPUT INPUTCLEAR INPUTCSV INPUTERR INPUTFIELD INS KEYEDIT KEYEXIT KEYTRAP LOCAL LOCATE LOCK LOGMSG LOOP MARK.MAPPING MAT MATBUILD MATPARSE MATREAD MATREADCSV MATREADL MATREADU MATWRITE MATWRITEU NAP NOBUF NULL OFF ON OPEN OPENPATH OPENSEQ OS.EXECUTE OSDELETE OSREAD OSWRITE PAGE PAUSE PERFORM PRECISION PRINT PRINTCSV PRINTER PRINTERR PRIVATE PROCREAD PROCWRITE PROGRAM PROMPT PUBLIC PUT RANDOMIZE READ READBLK READCSV READL READLIST READNEXT READSEQ READU READV READVL READVU RECORDLOCKL RECORDLOCKU RELEASE REMARK REMOVE REMOVE.BREAK.HANDLER REPEAT RESTORE.SCREEN RETURN ROLLBACK RQM SAVELIST SEEK SELECT SELECTE SELECTINDEX SELECTLEFT SELECTN SELECTRIGHT SELECTV SET.ARG SET.BREAK.HANDLER SET.EXIT.STATUS SET.TIMEZONE SETLEFT SETPU SETREM SETRIGHT SLEEP SSELECT START STATUS STOP STOPE STOPM SUBROUTINE TCLREAD THEN TIMEOUT TO TRANSACTION TTYSET UNLOCK UNTIL VOID WAKE WEOFSEQ WHILE WRITE WRITEBLK WRITECSV WRITESEQ WRITESEQF WRITEU WRITEV WRITEVU</Keywords>
<Keywords name="Words3">ABS ABSS ACCEPT.SOCKET.CONNECTION ACOS ALPHA ANDS ARG ARG.COUNT ARG.PRESENT ASCII ASIN ASSIGNED ATAN BINDKEY BITAND BITNOT BITOR BITRESET BITSET BITTEST BITXOR CATALOGUED CATS CHANGE CHAR CHECKSUM CHILD COL1 COL2 COMPARE COMPARES CONFIG CONNECT.PORT CONVERT COS COUNT COUNTS CREATE.SERVER.SOCKET CROP CSVDQ DATE DCOUNT DECRYPT DELETE DFPART DIR DIV DOWNCASE DQUOTE DQUOTES DTX EBCDIC ENCRYPT ENV EPOCH EQS EXP EXTRACT FCONTROL FIELD FIELDS FIELDSTORE FILE.EVENT FILEINFO FMT FMTS FOLD FOLDS GES GETARG.) GET.MESSAGES GET.PORT.PARAMS GETNLS GETPU GETREM GTS ICONV ICONVS IDIV IFS INDEX INDEXS INDICES INMAT INSERT INT ITYPE KEYCODE KEYIN KEYINC KEYINR KEYREADY LEN LENS LES LISTINDEX LN LOCATE LOWER LTS MATCHES MATCHESS MATCHFIELD MAX MAXIMUM MD5 MIN MINIMUM MOD MODS MVDATE MVDATE.TIME MVEPOCH MVTIME NEG NEGS NES NOT NOTS NUM NUMS OBJECT OBJINFO OCONV OCONVS OPEN.SOCKET ORS OS.ERROR OUTERJOIN PRINTER.SETTING PWR QUOTE QUOTES RAISE RDIV READ.SOCKET RECORDLOCKED REM REMOVE REMOVEF REPLACE REUSE RND ROUNDDOWN ROUNDUP RTRANS SAVE.SCREEN SELECTINFO SENTENCE SEQ SERVER.ADDR SET.PORT.PARAMS SET.SOCKET.MODE SETNLS SHIFT SIN SOCKET.INFO SOUNDEX SOUNDEXS SPACE SPACES SPLICE SQRT SQUOTE SQUOTES STATUS STR STRS SUBR SUBSTITUTE SUBSTRINGS SUM SUMMATION SWAP SWAPCASE SYSTEM TAN TERMINFO TESTLOCK TIME TIMEDATE TRANS TRIM TRIMB TRIMBS TRIMF TRIMFS TRIMS TTYGET UNASSIGNED UPCASE VOCPATH VSLICE WAIT.FILE.EVENT WRITE.SOCKET XLATE XTD</Keywords>
<Keywords name="Words4">@ABORT.CODE @ABORT.MESSAGE @ACCOUNT @AM @ANS @COMMAND @COMMAND.STACK @CONV @CRTHIGH @CRTWIDE @DATA @DATA.PENDING @DATE @DAY @DICTRECS @DS @FALSE @FILE.NAME @FILENAME @FM @FMT @GID @HOSTNAME @ID @IM @IP.ADDR @ITYPE.MODE @LEVEL @LOGNAME @LPTRHIGH @LPTRWIDE @MONTH @NB @NI @OPTION @PARASENTENCE @PATH @PIB @POB @QMSYS @RECORD @SELECTED @SENTENCE @SEQNO @SIB @SM @SOB @STDFIL @SVM @SYS.BELL @SYSTEM.RETURN.CODE @TERM.TYPE @TIME @TM @TRANSACTION.ID @TRANSACTION.LEVEL @TRIGGER.RETURN.CODE @TRUE @TTY @UID @USER @USER.NO @USER.RETURN.CODE @USER0 @USER1 @USER2 @USER3 @USER4 @USERNO @VM @VOC @WHO @YEAR @YEAR4</Keywords>
</KeywordLists>
<Styles>
<[[WordsStyle]] name="DEFAULT" styleID="11" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="FOLDEROPEN" styleID="12" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="FOLDERCLOSE" styleID="13" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="KEYWORD1" styleID="5" fgColor="[[FF00FF]]" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="KEYWORD2" styleID="6" fgColor="FF0000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="KEYWORD3" styleID="7" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="KEYWORD4" styleID="8" fgColor="800000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="COMMENT" styleID="1" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="COMMENT LINE" styleID="2" fgColor="00BF00" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="NUMBER" styleID="4" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="OPERATOR" styleID="10" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="DELIMINER1" styleID="14" fgColor="804000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="DELIMINER2" styleID="15" fgColor="804000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<[[WordsStyle]] name="DELIMINER3" styleID="16" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
</Styles>
</UserLang>
</NotepadPlus></pre>

== Universe ==
Following is a version for Universe basic. Click Language > basic to enable syntax highlighting.

<pre>
<NotepadPlus>
<UserLang name="basic" ext="">
<Settings>
<Global caseIgnored="yes" />
<TreatAsSymbol comment="no" commentLine="yes" />
<Prefix words1="no" words2="no" words3="no" words4="no" />
</Settings>
<KeywordLists>
<Keywords name="Delimiters">"'0"'0</Keywords>
<Keywords name="Folder+"></Keywords>
<Keywords name="Folder-"></Keywords>
<Keywords name="Operators">' - ! " # ( ) , / : ; [ ] ^ + < = ></Keywords>
<Keywords name="Comment">1 1 2 2 0* 0!</Keywords>
<Keywords name="Words1">$CHAIN $COPYRIGHT $DEFINE $EJECT $IFDEF $IFNDEF $INCLUDE $INSERT $MAP $OPTIONS $PAGE $UNDEFINE ABORT ABORTM ACTIVATEKEY AND AUTHORIZATION AUXMAP BEFORE BEGIN CASE TRANSACTION BREAK BSCAN CLEAR CLEARDATA CLEARFILE CLEARPROMPTS CLEARSELECT CLOSE CLOSESEQ COMMIT COMMON CONTINUE CONVERT CREATE CRT DATA DEACTIVATEKEY DEBUG DEFFUN DEL DELETE DELETELIST DIMENSION DIM DISABLEDEC DISPLAY DO ECHO ENABLEDEC END EOF ELSE EQUATE EQU ERRMSG EXIT FILELOCK FILEUNLOCK FIND FINDSTR FLUSH FOOTING FOR FORMLIST FROM FUNCTION GET GETX GETLIST GOSUB GOTO GO GROUPSTORE HEADING HUSH IF INCLUDE INPUT INPUTCLEAR INPUTDISP INPUTDP INPUTERR INPUTIF INPUTNULL INPUTTRAP INS ISOLATION KEYEDIT KEYEXIT KEYTRAP LET LIT LOCATE LOCK LOCKED LOOP MAT MATBUILD MATCH MATCHES MATPARSE MATREAD MATREADL MATREADU MATWRITE MATWRITEU NAP NEXT NOBUF NULL ON OPEN OPENCHECK OPENDEV OPENSEQ OR PAGE PRECISION PRINT PRINTER PRINTERR PROCREAD PROCWRITE PROGRAM PROMPT RANDOMIZE READ READBLK READL READLIST READNEXT READSEQ READT READU READV READVL READVU RECORDLOCK RELEASE REMOVE REPEAT RETURN REVREMOVE REWIND ROLLBACK SEEK SELECT SELECTE SELECTINDEX SEND SETREM SETTING SLEEP SSELECT STOP STOPM STORAGE SUBROUTINE SWAP TABSTOP THEN TIMEOUT TO TPRINT TTYCTL TTYGET TTYSET UNLOCK UPRINT UNTIL WAITING WEOF WEOFSEQ WHILE WRITE WRITEBLK WRITELIST WRITESEQ WRITESEQF WRITET WRITEU WRITEV WRITEVU LT LE EQ GE GT NE</Keywords>
<Keywords name="Words2">$FALSE $TRUE CALL CAPTURING CHAIN ENTER EXECUTE PERFORM RETURNING</Keywords>
<Keywords name="Words3">ABS ABSS acceptConnection ACOS addAuthenticationRule addCertificate addRequestParameter ADDS ALPHA amInitialize amReceiveMsg amReceiveRequest amSendMsg amSendRequest amSendResponse amTerminate analyzeCertificate ANDS ASCII ASIN ASSIGNED ATAN BITAND BITNOT BITOR BITRESET BITSET BITTEST BITXOR BYTE BYTELEN BYTETYPE BYTEVAL CATS CENTURY.PIVOT CHANGE CHAR CHARS CHECKSUM closeSocket CloseXMLData COL1 COL2 COMPARE COS COSH COUNT COUNTS createCertification createCertRequest createRequest createSecureRequest createSecurityContext DATE DBTOXML DCOUNT DESCRINFO DIGEST DIV DIVS DOWNCASE DQUOTE DTX EBCDIC ENCODE ENCRYPT EQS EREPLACE EXCHANGE EXP EXTRACT FADD FDIV FFIX FFLT FIELD FIELDS FIELDSTORE FILEINFO FIX FMT FMTDP FMTS FMTSDP FMUL FOLD FOLDDP FSUB generateKey GES getCipherSuite getSocketErrorMessage getSocketErrorMessage getSocketOptions getHTTPDefault GETLOCALE GETREM getSocketInformation GROUP GTS ICHECK ICONV ICONVS IFS ILPROMPT INDEX INDEXS INDICES initSecureServerSocket initSecureSocket INMAT INSERT INT ISNULL ISNULLS ITYPE KEYIN LEFT LEN LENDP LENS LENSDP LN loadSecurityContext LOCALEINFO LOWER LTS MATCHFIELD MAXIMUM MINIMUM MOD MODS MULS NEG NEGS NES NOT NOTS NUM NUMS OCONV OCONVS openSecureSocket openSocket OpenXMLData ORS PrePareXML protocolLogging PWR QUOTE RAISE readSocket ReadXMLData REAL RECORDLOCKED ReleaseXML REM REPLACE REUSE RIGHT RND RPC.CALL RPC.CONNECT RPC.DISCONNECT saveSecurityContext SADD SCMP SDIV SELECTINFO SENTENCE SEQ SEQS setAuthenticationDepth setCipherSuite setClientAuthentication setPrivateKey setRandomSeed setHTTPDefault setRequestHeader SETLOCALE setSocketOptions showSecurityContext SIGNATURE SIN SINH SMUL SOAPCreateRequest SOAPGetDefault SOAPGetFault SOAPGetResponseHeader SOAPSetRequestBody SOAPSetRequestContent SOAPSetRequestHeader SOAPRequestWrite SOAPSetDefault SOAPSetParameters SOAPSubmitRequest SOUNDEX SPACE SPACES SPLICE SQRT SQUOTE SSUB STATUS STR STRS submitRequest SUBR SUBS SUBSTRS SUM SUMMATION SYSTEM TAN TANH TERMINFO TIME TIMEDATE TPARM TRANS TRIM TRIMB TRIMBS TRIMF TRIMFS TRIMS UNASSIGNED UNICHAR UNISEQ UNISEQS UPCASE XLATE XTD</Keywords>
<Keywords name="Words4">@ABORT.CODE @ACCOUNT @AM @ANS @AUTHORIZARTION @COMMAND @COMMAND.STACK @CONV @CRTHIGH @CRTWIDE @DATA.PENDING @DATE @DAY @DICT @FALSE @FILE.NAME @FILENAME @FM @FORMAT @HDBC @HEADER @HENV @HSTMT @ID @IM @ISOLATION @LEVEL @LOGNAME @LPTRHIGH @LPTRWIDE @MONTH @MV @NB @ND @NEW @NI @NS @NULL @NULL.STR @NV @OLD @OPTION @PARASETENCE @PATH @RECOUNT @RECORD @RECUR0 @RECUR1 @RECUR2 @RECUR3 @RECUR4 @SCHEMA @SELECTED @SM @SQL.CODE @SQL.DATE @SQL.ERROR @SQL.STATE @SQL.TIME @SQL.WARNING @SQLPROC.NAME @SQLPROC.TSLEVEL @STDFIL @SVM @SYS.BELL @SYSTEM.RETURN.CODE @SYSTEM.SET @TERM.TYPE @TIME @TM @TRANSACTION @TRANSACTION.LEVEL @TRUE @TTY @USER0 @USER1 @USER2 @USER3 @USER4 @USERNO @USER.NO @USER.RETURN.CODE @VM @WHO @YEAR @YEAR4</Keywords>
</KeywordLists>
<Styles>
<WordsStyle name="DEFAULT" styleID="11" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" fontSize="10" />
<WordsStyle name="FOLDEROPEN" styleID="12" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="1" />
<WordsStyle name="FOLDERCLOSE" styleID="13" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="1" />
<WordsStyle name="KEYWORD1" styleID="5" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD2" styleID="6" fgColor="FF0080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="KEYWORD3" styleID="7" fgColor="0000FF" bgColor="FFFFFF" fontName="" fontStyle="0" fontSize="10" />
<WordsStyle name="KEYWORD4" styleID="8" fgColor="FF0000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="COMMENT" styleID="1" fgColor="008040" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="COMMENT LINE" styleID="2" fgColor="008000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="NUMBER" styleID="4" fgColor="FF0000" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="OPERATOR" styleID="10" fgColor="000080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER1" styleID="14" fgColor="800080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER2" styleID="15" fgColor="800080" bgColor="FFFFFF" fontName="" fontStyle="0" />
<WordsStyle name="DELIMINER3" styleID="16" fgColor="000000" bgColor="FFFFFF" fontName="" fontStyle="0" />
</Styles>
</UserLang>
</NotepadPlus>
</pre>

How-To Improve Large String Performance Using Buffers

2020-09-26T14:32:30Z

Rkgozar: /* Discussion */

[[Users]] >> [[Rex_Gozar]] >> How-To Improve Large String Performance Using Buffers
 

= How-To Improve Large String Performance Using Buffers =
2007-11-16 by Rex Gozar

== Brief ==
Processes that create or manipulate large amounts of data (over 1MB) may be slow, impacting user response time and system performance. This performance degradation can be traced to Universe needing to reallocate and copy memory repeatedly as the amount of data grows. If you can allocate all the memory that you'll need up front, you can eliminate this bottleneck. This document discusses a coding strategy for doing this.

== Summary ==
In processing a 12MB text file, execution time went from 83 seconds to 4 seconds by simply replacing VAR<-1> "append field" syntax with VAR[PTR,LENGTH] "substring assignment" syntax. Your mileage may vary.

<pre>
*** Using VAR<-1> "append field" syntax ***
ITEM = ""
LOOP
VALUE = some large amount of data
ITEM<-1> = VALUE
WHILE (SOME.CONDITION) REPEAT
</pre>

<pre>
*** Preallocating memory and using VAR[PTR,LENGTH] "substring assignment" syntax ***
MAXBYTES = some calculated value
BUFFER = SPACE(MAXBYTES) ;* allocates memory in one shot
BUFPTR = 0 ;* always points to last byte in buffer
LOOP
VALUE = some large amount of data
BUFFER[(1 + BUFPTR), (1 + LEN(VALUE))] = @FM:VALUE
BUFPTR += (1 + LEN(VALUE))
WHILE (SOME.CONDITION) REPEAT
BUFFER = BUFFER[2, (BUFPTR - 1)] ;* strip first @FM
</pre>

== Discussion ==
In my business application, I work with vendor-supplied text files that contain fixed-width fields. These files can be several megabytes in size. I created a subroutine to convert the fixed-widths fields for each row to tab separated values.

Initially, I used "append field" syntax (i.e. VAR<-1> = VALUE) to build the resulting TSV file. Running it took over 83 seconds -- ouch! I put in display statements to see what was going on, and I noticed that as more rows were processed, the slower and slower it got.

Now for the theoretical stuff: When we create a variable within a program, Universe has to allocate memory to hold the contents of the variable. When we append data to the variable, Universe has to see if the new data will exceed the amount of memory that it allocated. When it does, it has to ask the system for a new chunk of memory, then copy the contents of the old chunk to the new plus the new data. Large character strings tend to trigger this memory reallocation and copy over and over, slowing down the entire process. To keep this from happening, we need to allocate all the memory we need when we first initialize the variable.

The revised program is shown below. I create a NEWBUF that is twice the size of the original ITEM. In my case, I figured that would be big enough to prevent memory reallocation. Elapsed run time went from 83 seconds to 4, over 20 times faster.

<pre>
0001: SUBROUTINE FIXED.TO.TSV(ITEM, TABMAP)
0002: * Take a flat file of fixed-width values and insert
0003: * tabs so we don't have to code field widths in all
0004: * the other programs.
0005: ******
0006: ***
0007: * Set up a "macro" to get the system seconds
0008: * and milliseconds for performance timing.
0009: ***
0010: EQU GET$TICKS LIT '(SYSTEM(99):((SYSTEM(12) * 1000) "R%3"))'
0011: ***
0012: * Our TABMAP contains the field lengths (widths)
0013: * for each tab delimited field. Set the maximum
0014: * attribute mark count (MAXAMC).
0015: ***
0016: MAXAMC = DCOUNT(TABMAP, @FM)
0017: ***
0018: * Get the starting ticks, since we also want to
0019: * include memory allocation in our timings.
0020: ***
0021: START.TICKS = GET$TICKS
0022: ***
0023: * Initialize a variable in memory that's large enough
0024: * hold the existing data and the new tabs we'll be adding.
0025: ***
0026: NEWBUF = ITEM:ITEM
0027: NEWPTR = 0
0028: ***
0029: * Loop to remove each line for processing.
0030: ***
0031: ITEM = ITEM
0032: LOOP
0033: LINE = REMOVE(ITEM, MORE.LINES)
0034: GOSUB CONVERT.LINE
0035: ***
0036: * Note that the @FM is prepended to the
0037: * line, since we are not using <-1>
0038: * notation.
0039: ***
0040: NEWBUF[1+NEWPTR,1+LEN(LINE)] = @FM:LINE
0041: NEWPTR += (1 + LEN(LINE))
0042: WHILE MORE.LINES REPEAT
0043: ***
0044: * Strip off the leading @FM.
0045: ***
0046: ITEM = NEWBUF[2,NEWPTR-1]
0047: ***
0048: * Show the elapsed time in milliseconds.
0049: ***
0050: ELAPSED = GET$TICKS - START.TICKS
0051: DISPLAY ELAPSED
0052: RETURN
0053: *
0054: *
0055: CONVERT.LINE:
0056: WORKBUF = ""
0057: WORKPTR = 1
0058: FOR AMC = 1 TO MAXAMC
0059: FIELDLEN = TABMAP<AMC>
0060: WORKBUF<AMC> = LINE[WORKPTR, FIELDLEN]
0061: WORKPTR += FIELDLEN
0062: NEXT AMC
0063: LINE = CONVERT(@FM, CHAR(9), WORKBUF)
0064: RETURN
0065: END
</pre>

Could this subroutine be faster? Sure it can, but the point is all I did was one change and now it runs in 1/20th of the time.

== Recommendations ==
I only use this technique when performance can be noticeably increased. For example, building a savelist in a program may only be a few milliseconds faster using this technique, but who's going to notice? In many cases, readability and maintainability should take precedence over performance.

Always measure performance before and after optimizing your code. And only optimize the parts that take the longest to perform.

http://www.autopower.com/rgozar/pixel.gif

Talk:CodingStandards

2020-05-22T15:12:01Z

Rkgozar: wrapped code in code tag

== Ternary assignment preferred ==

Regarding 2.10 "Avoid using obscure and unfamiliar coding styles", the example shows ternary assignment as "BAD"; that is, obscure and unfamiliar to most PICK programmers. I actually prefer to use ternary assignment instead of multi-line IF-THEN-ELSE logic. Ternary assignment helps to avoid the error where the variable name is mis-typed in the ELSE clause.

== Passing an argument by value ==

Regarding 5.7 "Variables passed to subroutines can be passed by value", scalar variables can be wrapped in parentheses to pass by value, but an array element cannot. Calling <code>FMTPHONE((CM.REC(CM$PHONE)))</code> will pass a pointer reference to the array element, not a copy of its value, so the array element's value may be changed by the subroutine/function.

A better way to ensure an argument is passed by value is to prepend or append an empty string.

CALL FMTPHONE("":CM.REC(CM$PHONE))

This can be made more explicit with an equate:

EQU BYVAL LIT \ "": \
CALL FMTPHONE(BYVAL CM.REC(CM$PHONE))

Talk:CodingStandards

2020-05-15T14:20:12Z

Rkgozar:

== Ternary assignment preferred ==

Regarding 2.10 "Avoid using obscure and unfamiliar coding styles", the example shows ternary assignment as "BAD"; that is, obscure and unfamiliar to most PICK programmers. I actually prefer to use ternary assignment instead of multi-line IF-THEN-ELSE logic. Ternary assignment helps to avoid the error where the variable name is mis-typed in the ELSE clause.

== Passing an argument by value ==

Regarding 5.7 "Variables passed to subroutines can be passed by value", scalar variables can be wrapped in parentheses to pass by value, but an array element cannot. Calling FMTPHONE((CM.REC(CM$PHONE))) will pass a pointer reference to the array element, not a copy of its value, so the array element's value may be changed by the subroutine/function.

A better way to ensure an argument is passed by value is to prepend or append an empty string.

CALL FMTPHONE("":CM.REC(CM$PHONE))

This can be made more explicit with an equate:

EQU BYVAL LIT \ "": \
CALL FMTPHONE(BYVAL CM.REC(CM$PHONE))

Talk:CodingStandards

2020-05-15T14:19:23Z

Rkgozar: /* Passing an argument by value */ new section

Regarding 2.10 "Avoid using obscure and unfamiliar coding styles", the example shows ternary assignment as "BAD"; that is, obscure and unfamiliar to most PICK programmers. I actually prefer to use ternary assignment instead of multi-line IF-THEN-ELSE logic. Ternary assignment helps to avoid the error where the variable name is mis-typed in the ELSE clause.

== Passing an argument by value ==

Regarding 5.7 "Variables passed to subroutines can be passed by value", scalar variables can be wrapped in parentheses to pass by value, but an array element cannot. Calling FMTPHONE((CM.REC(CM$PHONE))) will pass a pointer reference to the array element, not a copy of its value, so the array element's value may be changed by the subroutine/function.

A better way to ensure an argument is passed by value is to prepend or append an empty string.

CALL FMTPHONE("":CM.REC(CM$PHONE))

This can be made more explicit with an equate:

EQU BYVAL LIT \ "": \
CALL FMTPHONE(BYVAL CM.REC(CM$PHONE))

Talk:CodingStandards

2020-05-15T14:07:28Z

Rkgozar: Ternary assignment preferred

Rex Gozar

2019-04-05T14:49:53Z

Rkgozar: removed pixel.gif that was used for tracking

[[Users]] >> Rex Gozar

"For every complex problem, there is a solution that is simple, neat, and wrong." H.L. Mencken

Contact me at [mailto:rgozar@gmail.com rgozar@gmail.com].

* [[QuickSort]] Universe BASIC implementation of the quick sort algorithm. 
* [[Row2Col]] Transpose a variable's "rows" (fields) into "columns" (values), and back again. 
* [[u2pipe]] Connect a web server to a Universe or Unidata database via u2pipe and wininetd. Ported to windows and unix. AIX, <nowiki>RedHat Linux</nowiki> and Windows binaries available. 
* [[How-To Improve Large String Performance Using Buffers]] Processes that create or manipulate large amounts of data (over 1MB) may be slow, impacting user response time and system performance. This performance degradation can be traced to Universe needing to reallocate and copy memory repeatedly as the amount of data grows. If you can allocate all the memory that you'll need up front, you can eliminate this bottleneck. This document discusses a coding strategy for doing this. 
* [[CodingStandards|Coding Standards]] Coding standards are basic rules to help us catch bugs and improve the quality of the programs we produce. They help us in all phases of development by giving us stable and time-tested coding structures that work. 
* [[Project Cleansweep]] I firmly believe that shoddy programming practices have hurt the credibility of Pick applications. I discuss active steps to conform your legacy application to modern coding standards. 
* [[Getting Personal with PickWiki]] I'm on a campaign to get every Pick developer to use [[PickWiki]]. Why? Because I know that there's a wealth of technical knowledge out there that would benefit the entire Pick community. But it's stupid for me to think that you brilliant technical innovators would contribute your precious time and effort purely for the betterment of mankind, nor for the sake of promoting the database we all know and love. Then I had a new thought... 
* [[CSV2FIELDS]] Convert a single CSV string into a field-delimited dynamic array. 
* [[Rex Gozar/Notes]] Various programming notes and half-baked ideas. Maybe I'll formalize them for public consumption (when I get the time -- HA!). 
* [[ENCRYPT64]] Sample code using the <tt>ENCRYPT</tt> function in Universe. 
* [[Hexadecimal Conversion]] Converting bytes into hexadecimal and back in Universe.

Accessing U2 From PHP

2019-04-05T14:39:58Z

Rkgozar: removed pixel.gif that was used for tracking

[[HowTos]] >> [[CreateWebInterfaces]] >> Accessing U2 from PHP

See also [[AccessFromPhp]] which includes other cross-platform ways to use PHP with MV. The techniques on this U2-specific page, and much of the code, can be used anywhere, and the code on the other page can augment solutions found here.

2008-01-14 from the u2ug users mailing list

== Rocket Articles on U2 and PHP ==
(from David Wolverton)

Look at this page for several articles on PHP and U2:

* https://u2devzone.rocketsoftware.com/

== PHP Web Services Strategy ==
(from Brian Leach)

PHP has good support for Web Services, so you could surface some UDT functionality that way. There was also an article on <nowiki>DeveloperWorks</nowiki> on
creating a custom PHP module that exposed some of the [[InterCall]] library, though it was a long way from a complete solution.

I also dimly remember someone writing an article on using a Java bridge to expose either RedBack objects or UOJ to PHP.

Some time ago I asked IBM whether they would be willing to publish the UniObjects protocol so that third parties could write clients for those environments it doesn't support, like PHP - but that wasn't too well received and I didn't follow up on it. Instead, I wrote a web services library (before IBM released theirs) and tested that through with PHP's SOAP support.

== Using the PHP COM library ==
(from Tony Gravagno)

With PHP5 you can just use the PHP COM library to access UO or UO.NET from PHP directly. It should be a LOT easier with PHP5 than PHP4. Here's an example that I've created after a brief reference to the UO doc (haven't tried to execute):

<pre>
<?php
$UOSession = new COM("UniObjects.Unioiafctrl")
or die("Cant instantiate UO");
$UOSession->HostName = "yourhost";
$UOSession->AccountPath = "d:\path\acct";
$UOSession->DatabaseType = 1; // UV here
$UOSession->UserName = ""; // *nix only?
$UOSession->Password = ""; // *nix only?
$UOSession->Connect; // need to check result
if ($UOSession->IsActive) {
$UOSession->Command->Text = "COUNT VOC"; // whatever
$UOSession->Command->Exec;
$U2Error = $UOSession->Command->Error;
$U2Status = $UOSession->Command->CommandStatus;
echo "Result is {$UOSession->Command->Response}";
// may need to loop on $UOSession->Command->NextBlock
}
$UOSession->Disconnect;
$UOSession = null;
?>
</pre>

This can do the same with UO.NET, mv.NET, and I'm guessing you can create an ADODB.Connection with a DSN and make SQL queries as well - all of this only over Windows. I believe FusionWare would have a similar Java-based solution for instantiating an object in PHP and accessing U2 entirely with <nowiki>*nix</nowiki>.

----
(from Steve Morley)

I converted our Admin system to work from the web via PHP5.

Install UNIDK on the webserver and put the following code inside a PHP page. This will initiate the connection to the Unidata server and allow you to talk via UniObjects.

<pre>
<?php
$Hostname = "192.168.0.1";
$Username = "admin";
$Password = "password";
$Accountpath = "c:\ibm\mydemoaccount";
$UdSession = new COM("UniObjects.unioaifctrl");
$UdSession->HostName = $Hostname;
$UdSession->AccountPath = $Accountpath;
$UdSession->UserName = $Username;
$UdSession->Password = $Password;
$UdSession->Connect();
?>
</pre>

----
(from Brian Leach)

COM is windows-specific (which is why I didn't think to suggest it for PHP.) If you want to use UniObjects on Linux you'll need to use UOJ (Java version).

== u2pipe ==
(from Rex Gozar)

I wrote [[u2pipe]] because I wanted (1) a single set of API subroutines on the U2 backend to handle all the business logic, and (2) to be able to port the API to different GUI clients (currently <nowiki>ColdFusion</nowiki> and PHP).

For our web application here, I created "web service subroutines" on the U2 backend, and wrote a PHP library of matching functions for the frontend to access them.

== Javascript and UniObjects ==
(Rick Brown)

We wrote <nowiki>JavaScript</nowiki> that basically uses <nowiki>UniObjects</nowiki> to connect to Unidata and run a subroutine. We pass data back to the PHP program in XML format. The <nowiki>JavaScript</nowiki> is just used as the connector, one input and one output. We have a few different <nowiki>UniBasic</nowiki> subroutines that can be called, one even allows the PHP developer to send a series of sentences in ECL format and receive the results in XML. The <nowiki>JavaScript</nowiki> is maybe 18 lines, and we run it on Tomcat.

This works well for getting smaller data sets to the Web, not so great for hundreds of records. Our PHP developer simply takes the returned XML and throws it into an array. With the same basic <nowiki>JavaScript</nowiki> we can pass the name of a subroutine to call, and provide input variables from PHP (Or pretty much any language with HTTP access)

Using HTTP as our transport for the XML is kind of a kludge, but has opened a lot of information to our web developers.

== Using U2 Web Development Environment ([[RedBack]]) ==

I wrote an object which allows full access to all the functionality in [[U2WDE]] from within PHP. This is a pure PHP script which requires no additional extensions to use. See https://github.com/gheydon/redback for the download and examples on how to use.

== Getting PHP / Java / Universe (Unidata) to Work Together ==

[[PhpJavaIntegration]]: as PHP does not as of this date support the Universe database, this is one approach to get data into your PHP application.

U2pipe

2016-10-10T13:32:00Z

Rkgozar: ini file syntax was escaped; now fixed for MediaWiki

[[Users]] >> [[Rex_Gozar]] >> u2pipe
 

= u2pipe / wininetd =

Version 1.0.3
 revised 2009-04-17 by Rex Gozar

== Brief ==

You can connect a web server to a Universe database via u2pipe. Windows platforms use wininetd as the socket listener. <nowiki>Unix/Linux</nowiki> platforms can use xinetd or inetd. This document explains how to do it.

There are better, more sophisticated ways of doing this. You could use <nowiki>RedBack</nowiki> (or whatever its new name is), <nowiki>UniObjects</nowiki>, mvNet, or any other commercial solution. But I'm tired of the "barriers to entry" in the PICK programming world, and I wanted to provide a simple open source alternative for connecting web servers to IBM U2 databases.

== Disclaimer ==

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

== Features ==

* Free (as in "free speech")
* Free (as in "free beer")
* Runs on Windows and unix.
* u2pipe - a single purpose utility to get the request from stdin, log into universe, and send the response to stdout.
* wininetd - an open source socket listening service
* Independent from web server; use any webserver/language you want -- ASP, JSP, PHP, <nowiki>ColdFusion</nowiki>, or whatever.
* Create as many socket ports as you want to access as many accounts as you want.
* Once a request is received, it can be handled by any Universe machine on your internal network.
* optional: You can restrict access to a list of "allowed" ip-addresses.
* optional: You can log every request and response.
* optional: You can (and should) run a filter program to validate TCL commands to execute.

== Non-Features ==

* not tested on Unidata, but uses [[InterCall]] so it should work.
* no connection pooling, but only eats a user license for the duration of the request processing.
* no connection limit, but you can create your own (when using xinetd, you can specify the number of instances; pending requests are queued).
* no encryption, but you can (and should) do your own.
* no multi-byte character support is built in, but since raw bytes are sent back and forth there's nothing stopping you from passing multi-byte characters.
* not responsible for character mangling by transport protocol. For example, HTTP is not guaranteed to transport binary characters as-is, but you can use url encoding, base64, or some other ASCII-armoring techniques for binary data.

== Binaries ==

* AIX: http://www.autopower.com/rgozar/u2pipe-1.0.3-aix.tgz
* Linux: http://www.autopower.com/rgozar/u2pipe-1.0.3-linux.tgz
* Windows: http://www.autopower.com/rgozar/u2pipe-1.0.3-setup.exe

== Windows Installation ==

# Download http://www.autopower.com/rgozar/u2pipe-1.0.3-setup.exe
# Run it.

* Must be logged in as an administrator so the wininetd service can be installed.
* Requires Windows 2000, or a later version.
* Requires access to Universe, either on the same machine, or reachable via <nowiki>UniRPC</nowiki>.
* No reboot required, so you can install anytime.
* The following files are installed.
** c:\windows\u2pipe.exe
** c:\windows\u2pipe.ini
** c:\windows\wininetd.exe
** c:\windows\wininetd.conf
** c:\windows\system32\unirpc32.dll
** c:\windows\system32\uvclnt32.dll
** c:\windows\system32\uvic32.dll

== AIX and Linux Installation ==
# You'll need to be savvy enough to install a new <nowiki>xinetd/inetd</nowiki> service (or at least know how to search Google.)
# Download the binary for your platform and untar it. Copy (or move) the <tt>u2pipe</tt> binary into the directory of your choice (typically <tt>/usr/local/bin</tt>).
# Edit <tt>/etc/services</tt> to add <tt>u2pipe</tt> with the port number of your choice. <pre>u2pipe 6789/tcp # u2pipe Web Service</pre>
# Edit your xinetd (or inetd) configuration file. AIX - sample inetd.conf<pre>u2pipe stream tcp nowait root /usr/local/bin/u2pipe</pre> Linux - sample /etc/xinetd.d/u2pipe<pre>service u2pipe
{
instances = 3
socket_type = stream
wait = no
user = root
server = /usr/local/bin/u2pipe
nice = 10
disable = no
only_from = localhost 10.0.0.0 192.168.1.132
}
</pre>
# Create the ini file. Use your favorite editor to create <tt>/etc/u2pipe.ini</tt> -- the format is described in "Configuring u2pipe.ini" below.
# Optional: You can configure a specific ini file to use by specifying the the "server_args". For example: <pre> server_args = /etc/sales_account.ini
</pre>

== Configuring wininetd ==

You typically don't have to change wininetd.conf, but if you want to the format is:

<tt>port user:password full-u2pipe-path [full-inifile-path]</tt>

* space or tab separators
* comment lines start with any non-numeric character
* <tt>port</tt> Any available port number.
* <tt>user:password</tt> A specific user id and password separated by a colon (:) or the word <tt>none</tt> to use the local system account (default).
* <tt>full-u2pipe-path</tt> Typically ''c:\windows\u2pipe.exe''
* <tt>full-inifile-path</tt> Optional: You can configure several u2pipe processes to run, each with their own ini file.
* sample: default configuration <tt>6789 none c:\windows\u2pipe.exe</tt>
* sample: using a different ini file <tt>9901 none c:\windows\u2pipe.exe c:\other.ini</tt>

In Windows ''Services'', you must change wininetd properties from "Manual" to "Automatic" for the service to start at reboot.

You must restart the wininetd service for changes to wininetd.conf to take effect. This is easy to do from the DOS command line by typing:

<tt>net stop wininetd</tt>
 <tt>net start wininetd</tt>

== Configuring u2pipe.ini ==

Changes to u2pipe.ini take effect immediately (does not require restart).

You can create different ini files to log into different machines or accounts, and tie them to specific port numbers in wininetd.conf (or inetd.conf, or xinetd.d files -- you get the idea).

* <tt>[U2PIPE]</tt> required: section declaration.
* <tt>HOSTNAME</tt> required: The Universe machine's hostname or ip-address.
* <tt>USER</tt> required: The user id to log into the machine.
* <tt>PASSWORD</tt> required: The user's password.
* <tt>ACCOUNT</tt> required: The Universe account to log into.
* <tt>RUN</tt> optional: Prefix the request with the specified string. For example, you may want to run a program named <tt>"MYPROG"</tt> to parse the request string for processing, so you would set <tt>"RUN=MYPROG"</tt>. The TCL command to execute will be <tt>"MYPROG whatever-the-request-string-is"</tt>. You should always run your requests through a program to parse arguments and filter out malicious text.
* <tt>ALLOWIP</tt> optional: List of allowed client IP-addresses that can make requests, separated by semi-colons (;). To allow requests from the CMD shell, i.e. <tt>ECH<nowiki>O</nowiki> 000004DATE|[[U2PIPE]]</tt>, add an extra semi-colon -- for example, <tt>ALLOWIP=192.168.1.7;</tt> (note the trailing semi-colon).
* <tt>LOGFILE</tt> optional: Log the requests and responses to a file. This file can grow big fast, so it is recommended that you only enable it for debugging. Also note that the log file is not locked and multiple u2pipe processes may update it simultaneously.
* <tt>;</tt> optional: Comments can be entered, or parameters no-op'ed, using a semicolon.

Sample <tt>u2pipe.ini</tt> with minimum configuration:
<pre>[U2PIPE]
hostname=localhost
user=bgates
password=iluvlinux!
account=SALES
</pre>

== Testing ==

Run both of these tests to make sure that u2pipe and wininetd are configured correctly. Error messages show up in the LOGFILE (specified in the ini file) or the Windows ''Event Viewer''.

* From the DOS command line type: <tt>ECH<nowiki>O</nowiki> 000004DATE|[[U2PIPE]]</tt>
** response: <tt>000037Friday, September 29, 2006 03:18pm</tt>
** Note that the response includes trailing CR and LF characters.

* From your browser, enter in the location: <tt><nowiki>http://yourhost:6789/000004DATE</nowiki></tt>
** response: <tt>000037Friday, September 29, 2006 03:18pm</tt>
** try: <tt><nowiki>http://yourhost:6789/000004TIME</nowiki></tt>
** try: <tt><nowiki>http://yourhost:6789/000005USERS</nowiki></tt>
** try: <tt><nowiki>http://yourhost:6789/000005LISTU</nowiki></tt>

* Note: Any single word TCL command can be performed. We'll need to do more work to allow multi-word commands.

* Note: Multi-line responses may get mangled by HTTP. We'll need to use ASCII armoring (i.e. base64 encoding) to reliably pass data back and forth.

* Note: Browsers and web utilities don't handle spaces very well, so at the very least we should use some kind of URL encoding. Other socket clients don't have this problem -- we can send and receive binary data without worrying about data corruption or transformation.

== Operation ==

''u2pipe'' is designed to take a request from stdin, process it, and send the response to stdout.

REQUEST format:

:<tt>123456command</tt>

where

:* <tt>123456</tt> is a six-digit request length. For example, the command <tt>DATE</tt> contains four characters; the request length is <tt>000004</tt>.

:* <tt>command</tt> is the TCL command to run. Ideally, the command should be encoded or encrypted so you can pass arguments for processing.

:* Example: <tt>000004DATE</tt>

''u2pipe'' reads in the request, strips off the six-digit request length, and prepends the optional RUN parameter (from the ini file above).

''u2pipe'' uses [[InterCall]] functions ic_opensession(), ic_execute(), and ic_quit() to process the command.
* @TTY is set to "uvcs" (for Universe; probably "udcs" on Unidata).
* The LOGIN paragraph is not run.
* Only one command is allowed (no LF or @FM separating commands).
* All characters are executed -- backspaces and such are not interpreted.
* Program output to the screen (via DISPLAY or CRT) is captured for the response.
* WARNING: CHAR(3) (CTRL-C) output breaks your Universe BASIC program, possibly truncating your output. Binary output should be encoded.

''u2pipe'' returns the length of the output and the output itself in the format of a response.

RESPONSE format:

:<tt>123456output</tt>

where

:* <tt>123456</tt> is a six-digit response length.

:* <tt>output</tt> is the output from your TCL command, including both printing and non-printing characters. Ideally, your output should be encoded or encrypted to protect the data from transformations and prying eyes.

:* Example: <tt>000037Friday, September 29, 2006 03:18pm</tt>

== Implementation ==

So you got the "DATE" command to work – Yahoo! break out the champagne (or maybe some of that "free beer")! No, wait a minute – I suppose you want to run something a little more complicated, huh?

* Maybe you want to run a subroutine, or a program that takes command line arguments.
* Maybe you want to run a TCL command that returns some XML.
* And some of you more security-minded folks out there want to know how encryption might fit into this.

I'm going to outline a ''simple'' implementation. You'll need to add your own features, like error return handling, beefier encryption, etc. The purpose of u2pipe is to provide a connection between a webserver and Universe; how to use that connection is up to ''you''.

1. We'll devise our own data format for exchanging information between the web server and Universe.

2. We'll write a function on the frontend.
* to format our request
* send it to the backend
* wait for the backend's response
* parse the response.

3. We'll write a Universe BASIC program on the backend
* to parse the request
* filter/validate the command
* run the command
* format the response
* return the response

4. We'll set the RUN parameter in u2pipe.ini to run our backend program.

5. We'll test it.

 <nowiki>1. Devising a data exchange format</nowiki>

This is a simple implementation, so I've decided that the webserver should be able to send TCL command strings with arguments and get the output in return. With that in mind, I've chosen to simply encode my command string and the output returned using base64. I've also decided to wrap the encoded output in parentheses "()" to help me find it in the response.

If I wanted to get fancy, I could encapsulate my request in XML (maybe like SOAP) and then encode, encrypt, or otherwise mangle the request before sending it to the backend. Conversely, I could also require the response to be XML formatted and encoded. Feel free to make it as simple or complicated as you wish.

 <nowiki>2. Writing a frontend function.</nowiki>

ASP, ASP.NET, JSP, PHP, <nowiki>ColdFusion</nowiki>... it does not matter what frontend you use. As long as your frontend can either (a) communicate on a socket, or (b) retrieve the contents of a web page, you can write a function for interacting with u2pipe.

 <nowiki>2a. Writing a frontend function: ColdFusion</nowiki>

I've already written a function for you. Change it as you need to implement your data exchange format.

<pre>

<cffunction name="decryptString" output="false" returntype="any">
<cfargument name="text" type="string" required="true">

<cfreturn toString(toBinary(text))>
</cffunction>

<cffunction name="encryptString" output="false" returntype="string">
<cfargument name="text" type="any" required="true">

<cfreturn toBase64(text, "us-ascii")>
</cffunction>

<cffunction name="serviceRequest" output="false" returntype="string">
<cfargument name="myCommand" type="string" required="true">

<cfset var HEADERSZ = 6>
<cfset var HOSTURL = "http://yourhost:6789">


<cfset var myLength = "">
<cfset var myRequest = "">
<cfset var myResponse = "">
<cfset var myOutput = "">
<cfset var bptr = 0>
<cfset var eptr = 0>


<cfset myCommand = encryptString(myCommand)>


<cfset myLength = right(repeatString("0", HEADERSZ) & len(myCommand), HEADERSZ)>
<cfset myRequest = HOSTURL & "/" & myLength & myCommand>
<cftry>
<cfhttp url="#myRequest#" timeout="60" />


<cfset myResponse = cfhttp.fileContent>
<cfset myLength = mid(myResponse, 1, HEADERSZ)>
<cfset myOutput = mid(myResponse, (HEADERSZ + 1), myLength)>



<cfset bptr = find("(", myOutput)>
<cfset eptr = find(")", myOutput)>
<cfif bptr lt eptr>
<cfset myOutput = mid(myOutput, (bptr+1), (eptr-bptr-1))>
<cfelse>
<cfthrow message="missing or invalid markers!">
</cfif>


<cfset myOutput = decryptString(myOutput)>

<cfcatch type="any">

</cfcatch>
</cftry>
<cfreturn myOutput>
</cffunction>


<cfset command = "LIST VOC SAMPLE 3">
<cfset output = serviceRequest(command)>
<cfoutput>
<nowiki><pre></nowiki>
#output#
<nowiki></pre></nowiki>
</cfoutput>
</pre>

 2b. Writing a frontend function: PHP

I've written a function for you PHP'ers too. Again, change it as you need to implement your data exchange format.

<pre>
<?php

function decryptString($text) {
// *TODO* substitute your own encryption
return base64_decode($text);
}

function encryptString($text) {
// *TODO* substitute your own encryption
return base64_encode($text);
}

function serviceRequest($myCommand) {

// set constants
define("HEADERSZ", 6);
define("HOSTIP", "yourhost");
define("HOSTPORT", 6789);

if (($socket = socket_create([[AF_INET]], [[SOCK_STREAM]], [[SOL_TCP]])) < 0 ) {
// *TODO* need to handle error"
die("Could not create socket!\n");
exit(1);
}

socket_set_option($socket, [[SOL_SOCKET]], [[SO_RCVTIMEO]], array('sec'=>20, 'usec'=>0));

if (!socket_connect($socket, HOSTIP, HOSTPORT)) {
// *TODO* need to handle error"
die("Unable to connect!\n");
exit(1);
}

$myCommand = encryptString($myCommand);

$myLength = strlen($myCommand);
$myRequest = sprintf("%0".HEADERSZ."d%s\n", $myLength, $myCommand);
socket_write($socket, $myRequest);

$myOutput = "";
$myLength = socket_read($socket, HEADERSZ);
while (FALSE != ($line = @socket_read($socket, $myLength, [[PHP_BINARY_READ]]))) {
$myOutput = $myOutput . $line;
}

$bptr = strpos($myOutput, "(");
$eptr = strpos($myOutput, ")");

//assert($bptr !=0);
//assert($eptr !=0);
//assert($bptr < $eptr);

$myOutput = substr($myOutput, ($bptr+1), ($eptr-$bptr-1));

$myOutput = decryptString($myOutput);
return ($myOutput);
}

$command = "LIST VOC SAMPLE 3";
$output = serviceRequest($command);

//header("Content-type: text/plain");
ech<nowiki>o "<pre>".$output."</pre>";</nowiki>
?>
</pre>

 <nowiki>3. Writing a backend Universe BASIC program</nowiki>

As I decided above, this backend program only needs to be able to base64 decode the command string and its arguments, and execute it. The output from the command is captured, cleaned up, and encoded to be returned to the frontend.

Note that all we have to do here is <tt>DISPLAY</tt> the response; <tt>u2pipe</tt> does the communication logic for us.

Let's name this program <tt>[[U2PIPERUN]]</tt>. Make sure you compile and catalog it in the appropriate account.

<pre>
0001: DISPLAY "!":
0002:
0003:
0004: DEFFUN DECODE64(TEXT)
0005: DEFFUN ENCODE64(TEXT)
0006: EQU LF TO CHAR(10)
0007: EQU FF TO CHAR(12)
0008: EQU CR TO CHAR(13)
0009:
0010:
0011: COMMAND = FIELD(@SENTENCE, " ", 2, LEN(@SENTENCE))
0012: COMMAND = DECODE64(COMMAND)
0013:
0014:
0015: EXECUTE COMMAND CAPTURING OUTPUT RETURNING RETCODE
0016:
0017:
0018: OUTPUT = CONVERT(@FM, LF, OUTPUT)
0019: OUTPUT = CONVERT(CR, "", OUTPUT)
0020: OUTPUT = TRIM(OUTPUT, FF, "B")
0021: OUTPUT = TRIM(OUTPUT, LF, "B")
0022: OUTPUT = ENCODE64(OUTPUT)
0023:
0024:
0025: DISPLAY "(":OUTPUT:")"
0026: END
Bottom at line 26.

</pre>

 4. Setting the <tt>RUN</tt> parameter in <tt>u2pipe.ini</tt>

Your <tt>u2pipe.ini</tt> file should look something like this:

<pre>
[U2PIPE]
HOSTNAME=yourhost
USER=webuser
PASSWORD=webuser_password
ACCOUNT=WEBACCOUNT
RUN=U2PIPERUN
</pre>

 5. Testing your implementation

You should be ready to test your implementation. You may want to turn on the <tt>LOGFILE</tt> parameter in the <tt>u2pipe.ini</tt> file to see the data being passed back and forth. You may also want to add some kind of error logging to your <tt>[[U2PIPERUN]]</tt> program.

 Other uses
* <tt>u2pipe</tt> could be used in windows BAT files (e.g. for scheduler routines). You could use bin\uvsh too, but uvsh cannot initiate jobs on other machines.

== Security ==

Obviously, you don't want just anyone sending TCL commands to your Universe database.

* Configure your firewall to only allow specific machines to access ''yourhost:6789''
* Set ALLOWIP in u2pipe.ini to only allow specific client ip-addresses.
* Encrypt your command in the request, and the output returned in the response. Universe has built-in encryption functions that you can use. You'll need some kind of encoding/encryption for all but the simplest commands anyway, so build it into your design/code.
* Create a unique user for logging in via u2pipe to your account, and restrict permissions.
* Lock down the Universe account accessed via u2pipe by:
** removing all verbs
** removing all paragraphs (procs, etc.)
** removing all file pointers (use OPENPATH)
** create an execution filter (set RUN in u2pipe.ini)
** only run subroutines from your execution filter

ALSO, <tt>wininetd</tt> can be used for evil; googling on it reveals that it can be used as part of a Trojan-virus attack. Some anti-virus software may report it as such. Just like <tt>telnet</tt> and <tt>ftp</tt>, <tt>wininetd</tt> is only as safe as your firewall setup!

== Design Decisions ==

Why write a socket-based solution instead of just using <nowiki>UniObjects</nowiki>? In my case, I needed to implement database connectivity for a remotely hosted web site; I couldn't "wire" <nowiki>UniObjects</nowiki> directly into the web server. And I really didn't want to.

Here are some decisions I made when writing this utility.
* I decided on a simple request/response model. I only wanted to create a pipe (stdin -> processing -> stdout). This is a requirement for wininetd.
* By using wininetd, I didn't have to worry about writing socket listener code, multi-threading, etc.
* Initially, I did not want a protocol (request and response length) but it made the memory allocation easier. It also allowed the client to easily check for the end of the response.
* I felt that encryption/decryption would have made the utility more complicated to write and would limit its flexibility. Most people have their own favorite crypto utilities anyway.
* I wanted the simplicity of Universe's [[InterCall]] API library. Most of the behavior is documented in the [[InterCall]] guide.

In designing your own web or socket solution, I think it is important to keep all your business logic on the database server. In some examples of <nowiki>UniObjects</nowiki> programming, I see business logic migrated or duplicated on the client - bad, bad, bad.

Whether you use <nowiki>UniObjects</nowiki> or u2pipe (or SQL for that matter), design your user interface to be database independent. Instead of embedding data access and updates directly in your web page (or GUI screen), put your database access logic in a separate function library. Then call those functions from your web page. This way, you'll be able to easily write test scaffolding to test your web page, then swap in the "real" functions that actually hit your database.

== Possible Enhancements ==

* You could use this as a base for developing an actual web service.
* Rewrite to use <nowiki>UniObjects</nowiki> instead of [[InterCall]].

== Downloading wininetd source code ==

You can download source code from http://xmailserver.org/wininetd.html

== u2pipe.c source code ==

* You must have <nowiki>UniDK</nowiki> installed on your development machine.
* Compiling notes
** See the [[InterCall]] guide for the DLL's you'll need.
** VC++ calling conventions need to be changed from __cdecl to __stdcall for the [[InterCall]] routines to link properly.
** Additional include directories: c:\ibm\unidk\include
** Additional lib directories: c:\ibm\unidk\lib
** Additional libraries/modules: uvic32.lib
* In the process of porting the code to run in unix, I replaced the Windows ini functions with the iniparser.h and iniparser.c source code libraries.

<pre>
/***
* u2pipe.c
* universe request pipe utility
* CVS $Revision: 1.3 $ $Date: 2007/05/11 19:33:27 $
*
* Copyright (C) 2006 Rex Gozar
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
* http://www.gnu.org/licenses/lgpl.html
*
* Rex Gozar
* rkgozar@juno.com
***/

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <intcall.h>
#include "iniparser.h"

#define MAXBUF (256)
#define MAXBIGBUF (8192)

#ifndef TRUE
#define TRUE (1)
#endif
#ifndef FALSE
#define FALSE (0)
#endif

#define [[APP_NAME]] ("[[U2PIPE]]")
#define [[DEFAULT_VALUE]] ("")
#define REQUIRED (TRUE)
#define [[NOT_REQUIRED]] (FALSE)

/***
* NOTE - GETSLASHSZ must be smaller than REQUESTLENSZ
* and (GETSLASHSZ + REQUESTLENSZ + 1) must be smaller than MAXBUF!
***/
#define GETSLASH ("GET /")
#define GETSLASHSZ (5)
#define REQUESTLENSZ (6)

/***
* helper functions
***/

char * get[[TimeStamp]](char *timestamp) {
time_t seconds = time(NULL);
struct tm *now = localtime(&seconds);
strftime(timestamp, MAXBUF, "%Y%m%d %H%M%S", now);
return timestamp;
}

/***
* main
*
* Note that the program should be compiled with __stdcall
* call convention so the intercall routines will properly link.
* Use __cdecl here to avoid a compiler warning.
***/

#ifndef WIN32
#define __cdecl
#endif

int __cdecl main(int argc, char *argv[]) {

time_t stime = time(NULL);
time_t etime = 0;

#ifdef WIN32
const char *WINDIR = getenv("WINDIR");
#else
const char *ETCDIR = "/etc";
#endif
const char *[[U2PIPE_INI]] = "u2pipe.ini";

long retcode = 0;
char buffer[MAXBUF] = "";
char timestamp[MAXBUF] = "";

const char *[[CLIENT_IP]] = getenv("[[CLIENT_IP]]");
const char *[[CLIENT_PORT]] = getenv("[[CLIENT_PORT]]");

char ipAddress[MAXBUF] = "";
char ipPort[MAXBUF] = "";
char iniFile[MAXBUF] = "";

dictionary *ini = NULL;
char *iniHostname = NULL;
char *iniUser = NULL;
char *iniPassword = NULL;
char *iniAccount = NULL;
char *iniRun = NULL;
char *iniLogfile = NULL;
char *ini[[AllowIP]] = NULL;

int logsw = FALSE;
FILE *fpLog;

int j = 0;

char *request = NULL;
long requestLen = 0;

char *response = NULL;
long responseLen = 0;
long responseMax = 0;

long sessionId = 0;
long code = 0;
long retcode2 = 0;

char *command = NULL;
long commandLen = 0;

/* check for NULL pointers and initialize ip address and port strings */

if (NULL != [[CLIENT_IP]]) {
strncpy(ipAddress, [[CLIENT_IP]], strlen([[CLIENT_IP]]));
}
if (NULL != [[CLIENT_PORT]]) {
strncpy(ipPort, [[CLIENT_PORT]], strlen([[CLIENT_PORT]]));
}

/* see if the ini file path is on the command line */

if (2 <= argc) {
if (MAXBUF < ( 1 + strlen(argv[1]) )) {
fprintf(stderr, "ini file pathname is longer than %i!\n", MAXBUF);
exit(1);
}
sprintf(iniFile, "%s", argv[1]);
} else {

#ifdef WIN32

if (NULL == WINDIR) {
fprintf(stderr, "WINDIR is NULL!\n");
exit(1);
}

/* build the default ini file path %WINDIR%\u2pipe.ini */
if (MAXBUF < ( 1 + strlen(WINDIR) + 1 + strlen([[U2PIPE_INI]]) )) {
fprintf(stderr, "%%WINDIR%%\\%s is longer than %i!\n", [[U2PIPE_INI]], MAXBUF);
exit(1);
}
sprintf(iniFile, "%s\\%s", WINDIR, [[U2PIPE_INI]]);

#else

if (MAXBUF < ( 1 + strlen(ETCDIR) + 1 + strlen([[U2PIPE_INI]]) )) {
fprintf(stderr, "%s/%s is longer than %i!\n", ETCDIR, [[U2PIPE_INI]], MAXBUF);
exit(1);
}
sprintf(iniFile, "%s/%s", ETCDIR, [[U2PIPE_INI]]);

#endif
}

/* parse config for user, password, account, etc */
ini = iniparser_new(iniFile);
iniHostname = iniparser_getstring(ini, "u2pipe:hostname", "");
iniUser = iniparser_getstring(ini, "u2pipe:user", "");
iniPassword = iniparser_getstring(ini, "u2pipe:password", "");
iniAccount = iniparser_getstring(ini, "u2pipe:account", "");
iniRun = iniparser_getstring(ini, "u2pipe:run", "");
iniLogfile = iniparser_getstring(ini, "u2pipe:logfile", "");
ini[[AllowIP]] = iniparser_getstring(ini, "u2pipe:allowIP", "");

#ifdef _DEBUG
fprintf(stderr, "iniFile=%s\n", iniFile);
fprintf(stderr, "ipAddress=%s\n", ipAddress);
fprintf(stderr, "ipPort=%s\n", ipPort);
fprintf(stderr, "iniFile=%s\n", iniFile);
fprintf(stderr, "iniHostname=%s\n", iniHostname);
fprintf(stderr, "iniUser=%s\n", iniUser);
fprintf(stderr, "iniPassword=%s\n", iniPassword);
fprintf(stderr, "iniAccount=%s\n", iniAccount);
fprintf(stderr, "iniRun=%s\n", iniRun);
fprintf(stderr, "iniLogfile=%s\n", iniLogfile);
fprintf(stderr, "ini[[AllowIP]]=%s\n", ini[[AllowIP]]);
#endif

/* open the log file */

logsw = (0 != strlen(iniLogfile));
if (logsw) {
fpLog = fopen(iniLogfile, "a+");
if (NULL == fpLog) {
fprintf(stderr, "unable to open log file %s\n", iniLogfile);
exit(1);
}
/* print a separator line for each request */
fprintf(fpLog, "****************************************\n");
}

/* log the requester */

if (logsw) {
fprintf(fpLog, "%s ipAddress=%s\n", get[[TimeStamp]](timestamp), ipAddress);
fprintf(fpLog, "%s ipPort=%s\n", get[[TimeStamp]](timestamp), ipPort);
}

/* check allowed ip-addresses */

if (0 != strlen(ini[[AllowIP]])) {

/* sandwich the list between semi-colons */
if (MAXBUF < 3 + strlen(ini[[AllowIP]])) {
if (logsw) {
fprintf(fpLog, "%s ALLOWIP list is too long!\n", get[[TimeStamp]](timestamp));
}
exit(1);
}
sprintf(buffer, ";%s;", ini[[AllowIP]]);
memset(ini[[AllowIP]], 0, MAXBUF);
strncpy(ini[[AllowIP]], buffer, strlen(buffer));

/* do the same to the ip address */
if (MAXBUF < 3 + strlen(ipAddress)) {
if (logsw) {
fprintf(fpLog, "%s ipAddress is too long!\n", get[[TimeStamp]](timestamp));
}
exit(1);
}
sprintf(buffer, ";%s;", ipAddress);

/* now look for the ipAddress within the allowed IP's */
if (NULL == strstr(ini[[AllowIP]], buffer)) {
if (logsw) {
fprintf(fpLog, "%s ipAddress is not allowed!\n", get[[TimeStamp]](timestamp));
}
exit(1);
}
}

/****
*
* requests are piped in via stdin, and must be formatted:
*
* GET /000009something
* or
* 000009something
*
* where:
* "GET /" is an optional literal string (typically passed by a web request.)
* "000009" is a six-digit request length.
* "something" is the actual request.
***/

/* look for "GET /" */

memset(buffer, 0, MAXBUF);
fread(buffer, sizeof(char), GETSLASHSZ, stdin);
if (0 == strncmp(buffer, GETSLASH, GETSLASHSZ)) {
/* discard the buffer contents and get REQUESTLENSZ more characters */
memset(buffer, 0, MAXBUF);
fread(buffer, sizeof(char), REQUESTLENSZ, stdin);
} else {
/* read in more characters to fill in the request length */
j = GETSLASHSZ;
while (j < REQUESTLENSZ) {
int c = fgetc(stdin);
if (EOF == c) {
if (logsw) {
fprintf(fpLog, "%s not enough characters in stdin!\n", get[[TimeStamp]](timestamp));
}
exit(1);
}
buffer[j] = c;
j++;
}
}

/* check the request length string */

if (REQUESTLENSZ != strlen(buffer)) {
if (logsw) {
fprintf(fpLog, "%s request length is not %i characters!\n", get[[TimeStamp]](timestamp), REQUESTLENSZ);
}
exit(1);
}

for (j = 0; j < REQUESTLENSZ; j++) {
if (!isdigit(buffer[j])) {
if (logsw) {
fprintf(fpLog, "%s request length is non-numeric!\n", get[[TimeStamp]](timestamp));
}
exit(1);
}
}

requestLen = atoi(buffer);
if (0 == requestLen) {
if (logsw) {
fprintf(fpLog, "%s request length is zero!\n", get[[TimeStamp]](timestamp));
}
exit(1);
}

/* get the memory to read in the rest of the request */

request = (char *) malloc(1 + requestLen);
if (NULL == request) {
if (logsw) {
fprintf(fpLog, "%s request malloc failed!\n", get[[TimeStamp]](timestamp));
}
exit(1);
}

/* read in the request */

memset(request, 0, 1+requestLen);
fread(request, sizeof(char), requestLen, stdin);

/* log the request */

if (logsw) {
fprintf(fpLog, "%s REQUEST=%s\n", get[[TimeStamp]](timestamp), request);
}

/*********************
* process the request
**********************/

sessionId = ic_opensession(iniHostname, iniUser, iniPassword, iniAccount, &code, NULL);
if (0 != code) {
if (logsw) {
fprintf(fpLog, "%s ic_opensession failed! %i\n", get[[TimeStamp]](timestamp), code);
}
exit(1);
}

if (0 != strlen(iniRun)) {
/* add one to the command length for the space between */
commandLen = strlen(iniRun) + 1 + requestLen;
command = (char *) malloc(1+commandLen);
memset(command, 0, commandLen);
sprintf(command, "%s %s", iniRun, request);
} else {
commandLen = requestLen;
command = (char *) malloc(1+commandLen);
memset(command, 0, commandLen);
sprintf(command, "%s", request);
}

responseMax = MAXBIGBUF;
response = (char *) malloc(responseMax);
memset(response, 0, responseMax);

ic_execute(command, &commandLen, response, &responseMax,
&responseLen, &retcode, &retcode2, &code);

if (logsw) {
fprintf(fpLog, "%s [[IC_EXECUTE]]=%i\n", get[[TimeStamp]](timestamp), code);
fprintf(fpLog, "%s responseLen=%i\n", get[[TimeStamp]](timestamp), responseLen);
}

/* see if we need to expand the response buffer to hold all the data */

while ([[IE_BTS]] == code) {

char *tmpbuf = NULL;
long tmpmax = 0;
long tmplen = 0;

char *newbuf = NULL;
long newmax = 0;

/* create a temporary buffer to hold the next block of data */
tmpmax = responseMax;
tmpbuf = (char *) malloc(tmpmax);
if (NULL == tmpbuf) {
fprintf(stderr, "malloc failed when creating tmpbuf!\n");
}
memset(tmpbuf, 0, tmpmax);
tmplen = 0;

/* get the next block of data */
ic_executecontinue(tmpbuf, &tmpmax, &tmplen, &retcode, &retcode2, &code);
if (logsw) {
fprintf(fpLog, "%s [[IC_EXECUTECONTINUE]]=%i\n", get[[TimeStamp]](timestamp), code);
}

/* create a new buffer to hold both the response and tmp buffers */
/* note that the new buffer is double the size of the previous iteration */
newmax = responseMax + tmpmax;
newbuf = (char *) malloc(newmax);
if (NULL == newbuf) {
fprintf(stderr, "malloc failed when creating newbuf!\n");
exit(1);
}
memset(newbuf, 0, newmax);

/* copy both the response and tmpbuf into the new buffer */
memcpy(newbuf, response, responseLen);
memcpy(newbuf + responseLen, tmpbuf, tmpmax);

/* release the memory for old data buffers and point response to the new data */
free(tmpbuf);
free(response);
response = newbuf;
responseMax = newmax;
responseLen = responseLen + tmplen;
}

ic_quit(&code);

/* log the response */
if (logsw) {
fprintf(fpLog, "%s RESPONSE=%s\n", get[[TimeStamp]](timestamp), response);
fprintf(fpLog, "%s responseLen=%i\n", get[[TimeStamp]](timestamp), responseLen);
fprintf(fpLog, "%s strlen(response)=%i\n", get[[TimeStamp]](timestamp), strlen(response));
}

/* return the response on stdout */

fprintf(stdout, "%06i%s\n", responseLen, response);
fflush(stdout);

/* done */
if (logsw) {
etime = time(NULL) - stime;
fprintf(fpLog, "%s COMPLETED (%lu)\n", get[[TimeStamp]](timestamp), etime);
fflush(fpLog);
fclose(fpLog);
}
exit(0);
}
</pre>

<pre>
/*
Based upon libiniparser, by Nicolas Devillard
Hacked into 1 file (m-iniparser) by Freek/2005
Original terms following:

-- -

Copyright (c) 2000 by Nicolas Devillard (ndevilla AT free DOT fr).

Written by Nicolas Devillard. Not derived from licensed software.

Permission is granted to anyone to use this software for any
purpose on any computer system, and to redistribute it freely,
subject to the following restrictions:

1. The author is not responsible for the consequences of use of
this software, no matter how awful, even if they arise
from defects in it.

2. The origin of this software must not be misrepresented, either
by explicit claim or by omission.

3. Altered versions must be plainly marked as such, and must not
be misrepresented as being the original software.

4. This notice may not be removed or altered.

*/

#ifndef _[[INIPARSER_H_]]
#define _[[INIPARSER_H_]]
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/* #include <unistd.h> */
#include <ctype.h>

#ifdef __cplusplus
extern "C" {
#endif

typedef struct _dictionary_ {
/** Number of entries in dictionary */
int n;
/** Storage size */
int size;
/** List of string values */
char **val;
/** List of string keys */
char **key ;
/** List of hash values for keys */
unsigned *hash;
} dictionary ;

/* generated by genproto */

dictionary * iniparser_new(char *ininame);
void iniparser_free(dictionary *d);

int iniparser_getnsec(dictionary *d);
char * iniparser_getsecname(dictionary *d, int n);
void iniparser_dump(dictionary *d, FILE *f);
void iniparser_dump_ini(dictionary *d, FILE *f);
char * iniparser_getkey(dictionary *d, char *section, char *key);
char * iniparser_getstr(dictionary *d, char *key);
char * iniparser_getstring(dictionary *d, char *key, char *def);
int iniparser_getint(dictionary *d, char *key, int notfound);
double iniparser_getdouble(dictionary *d, char *key, double notfound);
int iniparser_getboolean(dictionary *d, char *key, int notfound);
int iniparser_find_entry(dictionary *ini, char *entry);
int iniparser_setstr(dictionary *ini, char *entry, char *val);
void iniparser_unset(dictionary *ini, char *entry);

#ifdef __cplusplus
}
#endif

#endif
</pre>

<pre>

/*
Based upon libiniparser, by Nicolas Devillard
Hacked into 1 file (m-iniparser) by Freek/2005
Original terms following:

-- -

Copyright (c) 2000 by Nicolas Devillard (ndevilla AT free DOT fr).

Written by Nicolas Devillard. Not derived from licensed software.

Permission is granted to anyone to use this software for any
purpose on any computer system, and to redistribute it freely,
subject to the following restrictions:

1. The author is not responsible for the consequences of use of
this software, no matter how awful, even if they arise
from defects in it.

2. The origin of this software must not be misrepresented, either
by explicit claim or by omission.

3. Altered versions must be plainly marked as such, and must not
be misrepresented as being the original software.

4. This notice may not be removed or altered.

*/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/* #include <unistd.h> */

#include "iniparser.h"

#ifdef __cplusplus
extern "C" {
#endif

/* strlib.c following */

#define ASCIILINESZ 1024
/*-------------------------------------------------------------------------*/
/**
@brief Convert a string to lowercase.
@param s String to convert.
@return ptr to statically allocated string.

This function returns a pointer to a statically allocated string
containing a lowercased version of the input string. Do not free
or modify the returned string! Since the returned string is statically
allocated, it will be modified at each function call (not re-entrant).
*/
/*--------------------------------------------------------------------------*/

static char * strlwc(char *s)
{
static char l[ASCIILINESZ+1];
int i ;

if (s==NULL) return NULL ;
memset(l, 0, ASCIILINESZ+1);
i=0 ;
while (s[i] && i<ASCIILINESZ) {
l[i] = (char)tolower((int)s[i]);
i++ ;
}
l[ASCIILINESZ]=(char)0;
return l ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Convert a string to uppercase.
@param s String to convert.
@return ptr to statically allocated string.

This function returns a pointer to a statically allocated string
containing an uppercased version of the input string. Do not free
or modify the returned string! Since the returned string is statically
allocated, it will be modified at each function call (not re-entrant).
*/
/*--------------------------------------------------------------------------*/

static char * strupc(char *s)
{
static char l[ASCIILINESZ+1];
int i ;

if (s==NULL) return NULL ;
memset(l, 0, ASCIILINESZ+1);
i=0 ;
while (s[i] && i<ASCIILINESZ) {
l[i] = (char)toupper((int)s[i]);
i++ ;
}
l[ASCIILINESZ]=(char)0;
return l ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Skip blanks until the first non-blank character.
@param s String to parse.
@return Pointer to char inside given string.

This function returns a pointer to the first non-blank character in the
given string.
*/
/*--------------------------------------------------------------------------*/

static char * strskp(char *s)
{
char * skip = s;
if (s==NULL) return NULL ;
while (isspace((int)*skip) && *skip) skip++;
return skip ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Remove blanks at the end of a string.
@param s String to parse.
@return ptr to statically allocated string.

This function returns a pointer to a statically allocated string,
which is identical to the input string, except that all blank
characters at the end of the string have been removed.
Do not free or modify the returned string! Since the returned string
is statically allocated, it will be modified at each function call
(not re-entrant).
*/
/*--------------------------------------------------------------------------*/

static char * strcrop(char *s)
{
static char l[ASCIILINESZ+1];
char * last ;

if (s==NULL) return NULL ;
memset(l, 0, ASCIILINESZ+1);
strcpy(l, s);
last = l + strlen(l);
while (last > l) {
if (!isspace((int)*(last-1)))
break ;
last -- ;
}
*last = (char)0;
return l ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Remove blanks at the beginning and the end of a string.
@param s String to parse.
@return ptr to statically allocated string.

This function returns a pointer to a statically allocated string,
which is identical to the input string, except that all blank
characters at the end and the beg. of the string have been removed.
Do not free or modify the returned string! Since the returned string
is statically allocated, it will be modified at each function call
(not re-entrant).
*/
/*--------------------------------------------------------------------------*/
static char * strstrip(char *s)
{
static char l[ASCIILINESZ+1];
char * last ;

if (s==NULL) return NULL ;

while (isspace((int)*s) && *s) s++;

memset(l, 0, ASCIILINESZ+1);
strcpy(l, s);
last = l + strlen(l);
while (last > l) {
if (!isspace((int)*(last-1)))
break ;
last -- ;
}
*last = (char)0;

return (char*)l ;
}

/* dictionary.c.c following */
/** Maximum value size for integers and doubles. */
#define MAXVALSZ 1024

/** Minimal allocated number of entries in a dictionary */
#define DICTMINSZ 128

/** Invalid key token */
#define [[DICT_INVALID_KEY]] ((char*)-1)

/*
Doubles the allocated size associated to a pointer
'size' is the current allocated size.
*/
static void * mem_double(void *ptr, int size)
{
void *newptr;

newptr = calloc(2*size, 1);
memcpy(newptr, ptr, size);
free(ptr);
return newptr ;
}

/*---------------------------------------------------------------------------
Function codes
---------------------------------------------------------------------------*/

/*-------------------------------------------------------------------------*/
/**
@brief Compute the hash key for a string.
@param key Character string to use for key.
@return 1 unsigned int on at least 32 bits.

This hash function has been taken from an Article in Dr Dobbs Journal.
This is normally a collision-free function, distributing keys evenly.
The key is stored anyway in the struct so that collision can be avoided
by comparing the key itself in last resort.
*/
/*--------------------------------------------------------------------------*/

static unsigned dictionary_hash(char *key)
{
int len ;
unsigned hash ;
int i ;

len = strlen(key);
for (hash=0, i=0 ; i<len ; i++) {
hash += (unsigned)key[i] ;
hash += (hash<<10);
hash ^= (hash>>6) ;
}
hash += (hash <<3);
hash ^= (hash >>11);
hash += (hash <<15);
return hash ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Create a new dictionary object.
@param size Optional initial size of the dictionary.
@return 1 newly allocated dictionary objet.

This function allocates a new dictionary object of given size and returns
it. If you do not know in advance (roughly) the number of entries in the
dictionary, give size=0.
*/
/*--------------------------------------------------------------------------*/

static dictionary * dictionary_new(int size)
{
dictionary *d ;

/* If no size was specified, allocate space for DICTMINSZ */
if (size<DICTMINSZ) size=DICTMINSZ ;

d = (dictionary *)calloc(1, sizeof(dictionary));
d->size = size ;
d->val = (char **)calloc(size, sizeof(char*));
d->key = (char **)calloc(size, sizeof(char*));
d->hash = (unsigned int *)calloc(size, sizeof(unsigned));

return d;
}

/*-------------------------------------------------------------------------*/
/**
@brief Delete a dictionary object
@param d dictionary object to deallocate.
@return void

Deallocate a dictionary object and all memory associated to it.
*/
/*--------------------------------------------------------------------------*/

static void dictionary_del(dictionary *d)
{
int i ;

if (d==NULL) return ;
for (i=0 ; i<d->size ; i++) {
if (d->key[i]!=NULL)
free(d->key[i]);
if (d->val[i]!=NULL)
free(d->val[i]);
}
free(d->val);
free(d->key);
free(d->hash);
free(d);

return;
}

/*-------------------------------------------------------------------------*/
/**
@brief Get a value from a dictionary.
@param d dictionary object to search.
@param key Key to look for in the dictionary.
@param def Default value to return if key not found.
@return 1 pointer to internally allocated character string.

This function locates a key in a dictionary and returns a pointer to its
value, or the passed 'def' pointer if no such key can be found in
dictionary. The returned character pointer points to data internal to the
dictionary object, you should not try to free it or modify it.
*/
/*--------------------------------------------------------------------------*/
static char * dictionary_get(dictionary *d, char *key, char *def)
{
unsigned hash ;
int i ;

hash = dictionary_hash(key);
for (i=0 ; i<d->size ; i++) {
if (d->key==NULL)
continue ;
/* Compare hash */
if (hash==d->hash[i]) {
/* Compare string, to avoid hash collisions */
if (!strcmp(key, d->key[i])) {
return d->val[i] ;
}
}
}
return def ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Set a value in a dictionary.
@param d dictionary object to modify.
@param key Key to modify or add.
@param val Value to add.
@return void

If the given key is found in the dictionary, the associated value is
replaced by the provided one. If the key cannot be found in the
dictionary, it is added to it.

It is Ok to provide a NULL value for val, but NULL values for the dictionary
or the key are considered as errors: the function will return immediately
in such a case.

Notice that if you dictionary_set a variable to NULL, a call to
dictionary_get will return a NULL value: the variable will be found, and
its value (NULL) is returned. In other words, setting the variable
content to NULL is equivalent to deleting the variable from the
dictionary. It is not possible (in this implementation) to have a key in
the dictionary without value.
*/
/*--------------------------------------------------------------------------*/

static void dictionary_set(dictionary *d, char *key, char *val)
{
int i ;
unsigned hash ;

if (d==NULL || key==NULL) return ;

/* Compute hash for this key */
hash = dictionary_hash(key) ;
/* Find if value is already in blackboard */
if (d->n>0) {
for (i=0 ; i<d->size ; i++) {
if (d->key[i]==NULL)
continue ;
if (hash==d->hash[i]) { /* Same hash value */
if (!strcmp(key, d->key[i])) { /* Same key */
/* Found a value: modify and return */
if (d->val[i]!=NULL)
free(d->val[i]);
d->val[i] = val ? strdup(val) : NULL ;
/* Value has been modified: return */
return ;
}
}
}
}
/* Add a new value */
/* See if dictionary needs to grow */
if (d->n==d->size) {

/* Reached maximum size: reallocate blackboard */
d->val = (char **)mem_double(d->val, d->size * sizeof(char*)) ;
d->key = (char **)mem_double(d->key, d->size * sizeof(char*)) ;
d->hash = (unsigned int *)mem_double(d->hash, d->size * sizeof(unsigned)) ;

/* Double size */
d->size *= 2 ;
}

/* Insert key in the first empty slot */
for (i=0 ; i<d->size ; i++) {
if (d->key[i]==NULL) {
/* Add key here */
break ;
}
}
/* Copy key */
d->key[i] = strdup(key);
d->val[i] = val ? strdup(val) : NULL ;
d->hash[i] = hash;
d->n ++ ;
return ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Delete a key in a dictionary
@param d dictionary object to modify.
@param key Key to remove.
@return void

This function deletes a key in a dictionary. Nothing is done if the
key cannot be found.
*/
/*--------------------------------------------------------------------------*/
static void dictionary_unset(dictionary *d, char *key)
{
unsigned hash ;
int i ;

hash = dictionary_hash(key);
for (i=0 ; i<d->size ; i++) {
if (d->key[i]==NULL)
continue ;
/* Compare hash */
if (hash==d->hash[i]) {
/* Compare string, to avoid hash collisions */
if (!strcmp(key, d->key[i])) {
/* Found key */
break ;
}
}
}
if (i>=d->size)
/* Key not found */
return ;

free(d->key[i]);
d->key[i] = NULL ;
if (d->val[i]!=NULL) {
free(d->val[i]);
d->val[i] = NULL ;
}
d->hash[i] = 0 ;
d->n -- ;
return ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Dump a dictionary to an opened file pointer.
@param d Dictionary to dump
@param f Opened file pointer.
@return void

Dumps a dictionary onto an opened file pointer. Key pairs are printed out
as @c [Key]=[Value], one per line. It is Ok to provide stdout or stderr as
output file pointers.
*/
/*--------------------------------------------------------------------------*/

static void dictionary_dump(dictionary *d, FILE *f)
{
int i;

if (d==NULL || f==NULL) return;

for (i=0; i<d->size; i++) {
if (d->key[i] == NULL)
continue ;
if (d->val[i] != NULL) {
fprintf(f, "[%s]=[%s]\n", d->key[i], d->val[i]);
} else {
fprintf(f, "[%s]=UNDEF\n", d->key[i]);
}
}

return;
}

/* iniparser.c.c following */
#define ASCIILINESZ 1024
#define [[INI_INVALID_KEY]] ((char*)-1)

/* Private: add an entry to the dictionary */
static void iniparser_add_entry(
dictionary *d,
char *sec,
char *key,
char *val)
{
char longkey[2*ASCIILINESZ+1];

/* Make a key as section:keyword */
if (key!=NULL) {
sprintf(longkey, "%s:%s", sec, key);
} else {
strcpy(longkey, sec);
}

/* Add (key,val) to dictionary */
dictionary_set(d, longkey, val);
return ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Get number of sections in a dictionary
@param d Dictionary to examine
@return int Number of sections found in dictionary

This function returns the number of sections found in a dictionary.
The test to recognize sections is done on the string stored in the
dictionary: a section name is given as "section" whereas a key is
stored as "section:key", thus the test looks for entries that do not
contain a colon.

This clearly fails in the case a section name contains a colon, but
this should simply be avoided.

This function returns -1 in case of error.
*/
/*--------------------------------------------------------------------------*/

int iniparser_getnsec(dictionary *d)
{
int i ;
int nsec ;

if (d==NULL) return -1 ;
nsec=0 ;
for (i=0 ; i<d->size ; i++) {
if (d->key[i]==NULL)
continue ;
if (strchr(d->key[i], ':')==NULL) {
nsec ++ ;
}
}
return nsec ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Get name for section n in a dictionary.
@param d Dictionary to examine
@param n Section number (from 0 to nsec-1).
@return Pointer to char string

This function locates the n-th section in a dictionary and returns
its name as a pointer to a string statically allocated inside the
dictionary. Do not free or modify the returned string!

This function returns NULL in case of error.
*/
/*--------------------------------------------------------------------------*/

char * iniparser_getsecname(dictionary *d, int n)
{
int i ;
int foundsec ;

if (d==NULL || n<0) return NULL ;
foundsec=0 ;
for (i=0 ; i<d->size ; i++) {
if (d->key[i]==NULL)
continue ;
if (strchr(d->key[i], ':')==NULL) {
foundsec++ ;
if (foundsec>n)
break ;
}
}
if (foundsec<=n) {
return NULL ;
}
return d->key[i] ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Dump a dictionary to an opened file pointer.
@param d Dictionary to dump.
@param f Opened file pointer to dump to.
@return void

This function prints out the contents of a dictionary, one element by
line, onto the provided file pointer. It is OK to specify @c stderr
or @c stdout as output files. This function is meant for debugging
purposes mostly.
*/
/*--------------------------------------------------------------------------*/
void iniparser_dump(dictionary *d, FILE *f)
{
dictionary_dump(d,f);
}

/*-------------------------------------------------------------------------*/
/**
@brief Save a dictionary to a loadable ini file
@param d Dictionary to dump
@param f Opened file pointer to dump to
@return void

This function dumps a given dictionary into a loadable ini file.
It is Ok to specify @c stderr or @c stdout as output files.
*/
/*--------------------------------------------------------------------------*/

void iniparser_dump_ini(dictionary *d, FILE *f)
{
int i, j ;
char keym[ASCIILINESZ+1];
int nsec ;
char * secname ;
int seclen ;

if (d==NULL || f==NULL) return ;

nsec = iniparser_getnsec(d);
if (nsec<1) {
/* No section in file: dump all keys as they are */
for (i=0 ; i<d->size ; i++) {
if (d->key[i]==NULL)
continue ;
fprintf(f, "%s = %s\n", d->key[i], d->val[i]);
}
return ;
}
for (i=0 ; i<nsec ; i++) {
secname = iniparser_getsecname(d, i) ;
seclen = (int)strlen(secname);
fprintf(f, "\n[%s]\n", secname);
sprintf(keym, "%s:", secname);
for (j=0 ; j<d->size ; j++) {
if (d->key[j]==NULL)
continue ;
if (!strncmp(d->key[j], keym, seclen+1)) {
fprintf(f,
"%-30s = %s\n",
d->key[j]+seclen+1,
d->val[j] ? d->val[j] : "");
}
}
}
fprintf(f, "\n");
return ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Get the string associated to a key, return NULL if not found
@param d Dictionary to search
@param key Key string to look for
@return pointer to statically allocated character string, or NULL.

This function queries a dictionary for a key. A key as read from an
ini file is given as "section:key". If the key cannot be found,
NULL is returned.
The returned char pointer is pointing to a string allocated in
the dictionary, do not free or modify it.

This function is only provided for backwards compatibility with
previous versions of iniparser. It is recommended to use
iniparser_getstring() instead.
*/
/*--------------------------------------------------------------------------*/
char * iniparser_getstr(dictionary *d, char *key)
{
return iniparser_getstring(d, key, NULL);
}

/*-------------------------------------------------------------------------*/
/**
@brief Get the string associated to a key
@param d Dictionary to search
@param key Key string to look for
@param def Default value to return if key not found.
@return pointer to statically allocated character string

This function queries a dictionary for a key. A key as read from an
ini file is given as "section:key". If the key cannot be found,
the pointer passed as 'def' is returned.
The returned char pointer is pointing to a string allocated in
the dictionary, do not free or modify it.
*/
/*--------------------------------------------------------------------------*/
char * iniparser_getstring(dictionary *d, char *key, char *def)
{
char * lc_key ;
char * sval ;

if (d==NULL || key==NULL)
return def ;

lc_key = strdup(strlwc(key));
sval = dictionary_get(d, lc_key, def);
free(lc_key);
return sval ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Get the string associated to a key, convert to an int
@param d Dictionary to search
@param key Key string to look for
@param notfound Value to return in case of error
@return integer

This function queries a dictionary for a key. A key as read from an
ini file is given as "section:key". If the key cannot be found,
the notfound value is returned.
*/
/*--------------------------------------------------------------------------*/
int iniparser_getint(dictionary *d, char *key, int notfound)
{
char *str ;

str = iniparser_getstring(d, key, [[INI_INVALID_KEY]]);
if (str==[[INI_INVALID_KEY]]) return notfound ;
return atoi(str);
}

/*-------------------------------------------------------------------------*/
/**
@brief Get the string associated to a key, convert to a double
@param d Dictionary to search
@param key Key string to look for
@param notfound Value to return in case of error
@return double

This function queries a dictionary for a key. A key as read from an
ini file is given as "section:key". If the key cannot be found,
the notfound value is returned.
*/
/*--------------------------------------------------------------------------*/
double iniparser_getdouble(dictionary *d, char *key, double notfound)
{
char * str ;

str = iniparser_getstring(d, key, [[INI_INVALID_KEY]]);
if (str==[[INI_INVALID_KEY]]) return notfound ;
return atof(str);
}

/*-------------------------------------------------------------------------*/
/**
@brief Get the string associated to a key, convert to a boolean
@param d Dictionary to search
@param key Key string to look for
@param notfound Value to return in case of error
@return integer

This function queries a dictionary for a key. A key as read from an
ini file is given as "section:key". If the key cannot be found,
the notfound value is returned.

A true boolean is found if one of the following is matched:

- A string starting with 'y'
- A string starting with 'Y'
- A string starting with 't'
- A string starting with 'T'
- A string starting with '1'

A false boolean is found if one of the following is matched:

- A string starting with 'n'
- A string starting with 'N'
- A string starting with 'f'
- A string starting with 'F'
- A string starting with '0'

The notfound value returned if no boolean is identified, does not
necessarily have to be 0 or 1.
*/
/*--------------------------------------------------------------------------*/
int iniparser_getboolean(dictionary *d, char *key, int notfound)
{
char *c ;
int ret ;

c = iniparser_getstring(d, key, [[INI_INVALID_KEY]]);
if (c==[[INI_INVALID_KEY]]) return notfound ;
if (c[0]=='y' || c[0]=='Y' || c[0]=='1' || c[0]=='t' || c[0]=='T') {
ret = 1 ;
} else if (c[0]=='n' || c[0]=='N' || c[0]=='0' || c[0]=='f' || c[0]=='F') {
ret = 0 ;
} else {
ret = notfound ;
}
return ret;
}

/*-------------------------------------------------------------------------*/
/**
@brief Finds out if a given entry exists in a dictionary
@param ini Dictionary to search
@param entry Name of the entry to look for
@return integer 1 if entry exists, 0 otherwise

Finds out if a given entry exists in the dictionary. Since sections
are stored as keys with NULL associated values, this is the only way
of querying for the presence of sections in a dictionary.
*/
/*--------------------------------------------------------------------------*/

int iniparser_find_entry(
dictionary *ini,
char *entry
)
{
int found=0 ;
if (iniparser_getstring(ini, entry, [[INI_INVALID_KEY]])!=[[INI_INVALID_KEY]]) {
found = 1 ;
}
return found ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Set an entry in a dictionary.
@param ini Dictionary to modify.
@param entry Entry to modify (entry name)
@param val New value to associate to the entry.
@return int 0 if Ok, -1 otherwise.

If the given entry can be found in the dictionary, it is modified to
contain the provided value. If it cannot be found, -1 is returned.
It is Ok to set val to NULL.
*/
/*--------------------------------------------------------------------------*/

int iniparser_setstr(dictionary *ini, char *entry, char *val)
{
dictionary_set(ini, strlwc(entry), val);
return 0 ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Delete an entry in a dictionary
@param ini Dictionary to modify
@param entry Entry to delete (entry name)
@return void

If the given entry can be found, it is deleted from the dictionary.
*/
/*--------------------------------------------------------------------------*/
void iniparser_unset(dictionary *ini, char *entry)
{
dictionary_unset(ini, strlwc(entry));
}

/*-------------------------------------------------------------------------*/
/**
@brief Parse an ini file and return an allocated dictionary object
@param ininame Name of the ini file to read.
@return Pointer to newly allocated dictionary

This is the parser for ini files. This function is called, providing
the name of the file to be read. It returns a dictionary object that
should not be accessed directly, but through accessor functions
instead.

The returned dictionary must be freed using iniparser_free().
*/
/*--------------------------------------------------------------------------*/

dictionary * iniparser_new(char *ininame)
{
dictionary *d ;
char lin[ASCIILINESZ+1];
char sec[ASCIILINESZ+1];
char key[ASCIILINESZ+1];
char val[ASCIILINESZ+1];
char *where ;
FILE *ini ;
int lineno ;

if ((ini=fopen(ininame, "r"))==NULL) {
return NULL ;
}

sec[0]=0;

/*
* Initialize a new dictionary entry
*/
d = dictionary_new(0);
lineno = 0 ;
while (fgets(lin, ASCIILINESZ, ini)!=NULL) {
lineno++ ;
where = strskp(lin); /* Skip leading spaces */
if (*where==';' || *where=='#' || *where==0)
continue ; /* Comment lines */
else {
if (sscanf(where, "[%[^]]", sec)==1) {
/* Valid section name */
strcpy(sec, strlwc(sec));
iniparser_add_entry(d, sec, NULL, NULL);
} else if (sscanf (where, "%[^=] = \"%[^\"]\"", key, val) == 2
|| sscanf (where, "%[^=] = '%[^\']'", key, val) == 2
|| sscanf (where, "%[^=] = %[^;#]", key, val) == 2) {
strcpy(key, strlwc(strcrop(key)));
/*
* sscanf cannot handle "" or '' as empty value,
* this is done here
*/
if (!strcmp(val, "\"\"") || !strcmp(val, "''")) {
val[0] = (char)0;
} else {
strcpy(val, strcrop(val));
}
iniparser_add_entry(d, sec, key, val);
}
}
}
fclose(ini);
return d ;
}

/*-------------------------------------------------------------------------*/
/**
@brief Free all memory associated to an ini dictionary
@param d Dictionary to free
@return void

Free all memory associated to an ini dictionary.
It is mandatory to call this function before the dictionary object
gets out of the current context.
*/
/*--------------------------------------------------------------------------*/

void iniparser_free(dictionary *d)
{
dictionary_del(d);
}

#ifdef __cplusplus
}
#endif

</pre>

== LGPL License ==

http://www.gnu.org/licenses/lgpl.html

== Revisions ==

* 2006-10-05: Initial version 1.0.0.
* 2007-03-13: Version 1.0.1 fixes an installer issue where wininetd would not be installed as a service when the install path did not default to "c:\windows" for the user. Wininetd can still be installed manually by typing <tt>wininetd --install</tt>.
* 2007-04-14: http://www.autopower.com/rgozar/u2pipe-1.0.1-setup.exe was hanging when clicked. Now fixed.
* 2007-08-27: Version 1.0.3 now ported for unix (AIX and <nowiki>RedHat</nowiki> Linux binaries hyperlinked above).
* 2009-04-17: Revised linux installation notes.

User talk:Rkgozar

2015-04-22T17:58:26Z

Rkgozar: group dynamics, anyone?

== Group Dynamics ==
Today I look at PickWiki's recent changes and all I see are a bunch of bot sign-ups; I can't see what's changed or newly contributed. A few weeks ago when the site first came up I thought I would try to make a small change to my own page (or as it was pointed out I don't really have my own page, just one that happens to have the same name as me) and weeks afterward the moderator still hadn't allowed the change. Only after making a side comment in the email list did my change get approved. As far as I can tell I'm the only contributor so far that's been willing to overlook the annoyances and attempt to make changes.

Group dynamics are vital to the survival of a community website. The smallest things can help a group to grow, or instantly kill it. I'm concerned that implementation decisions are being made that are actually hurting growth. What's being done to PickWiki with the specific intention of creating growth?

I'm not talking about software changes; MediaWiki is great software with millions of users. I'm talking about what's being done that will get that Pick professional with 30 years experience to ''want'' to contribute? And not drive him away? [[User:Rkgozar|Rkgozar]] ([[User talk:Rkgozar|talk]]) 10:58, 22 April 2015 (PDT)

User:Rkgozar

2015-04-13T12:54:28Z

Rkgozar: Created page with "Rex Gozar "For every complex problem, there is a solution that is simple, neat, and wrong." H.L. Mencken Contact me at [mailto:rgozar@gmail.com rgozar@gmail.com]. *..."

Rex Gozar

"For every complex problem, there is a solution that is simple, neat, and wrong." H.L. Mencken

Contact me at [mailto:rgozar@gmail.com rgozar@gmail.com].

* [[QuickSort]] Universe BASIC implementation of the quick sort algorithm. 
* [[Row2Col]] Transpose a variable's "rows" (fields) into "columns" (values), and back again. 
* [[u2pipe]] Connect a web server to a Universe or Unidata database via u2pipe and wininetd. Ported to windows and unix. AIX, <nowiki>RedHat Linux</nowiki> and Windows binaries available. 
* [[How-To Improve Large String Performance Using Buffers]] Processes that create or manipulate large amounts of data (over 1MB) may be slow, impacting user response time and system performance. This performance degradation can be traced to Universe needing to reallocate and copy memory repeatedly as the amount of data grows. If you can allocate all the memory that you'll need up front, you can eliminate this bottleneck. This document discusses a coding strategy for doing this. 
* [[CodingStandards|Coding Standards]] Coding standards are basic rules to help us catch bugs and improve the quality of the programs we produce. They help us in all phases of development by giving us stable and time-tested coding structures that work. 
* [[Project Cleansweep]] I firmly believe that shoddy programming practices have hurt the credibility of Pick applications. I discuss active steps to conform your legacy application to modern coding standards. 
* [[Getting Personal with PickWiki]] I'm on a campaign to get every Pick developer to use [[PickWiki]]. Why? Because I know that there's a wealth of technical knowledge out there that would benefit the entire Pick community. But it's stupid for me to think that you brilliant technical innovators would contribute your precious time and effort purely for the betterment of mankind, nor for the sake of promoting the database we all know and love. Then I had a new thought... 
* [[CSV2FIELDS]] Convert a single CSV string into a field-delimited dynamic array. 
* [[Rex Gozar/Notes]] Various programming notes and half-baked ideas. Maybe I'll formalize them for public consumption (when I get the time -- HA!). 
* [[ENCRYPT64]] Sample code using the <tt>ENCRYPT</tt> function in Universe. 
* [[Hexadecimal Conversion]] Converting bytes into hexadecimal and back in Universe.