MBUTIL - RemoteAccess/QuickBBS Message Base Utility 1.10 Copyright (C) 1990 Gerard van der

---
Master Index Current Directory Index Go to SkepticTank Go to Human Rights activist Keith Henson Go to Scientology cult

Skeptic Tank!

MBUTIL - RemoteAccess/QuickBBS Message Base Utility 1.10 Copyright (C) 1990 Gerard van der Land. All rights reserved. Features -------- * Sorting messages by board and post date and time * Purging messages by number, date or received status * Message base packing with AutoRenumber * Reply chain linking and subject cleaning * Creating index files and fixing all kinds of problems and errors * Undeleting deleted messages * Moving all messages in one board to another board * Posting a file as a message * Netmail Import .MSG ==> MSG*.BBS * Netmail Export MSG*.BBS ==> .MSG (updates credit) * Replacing up to 5 different tearlines in outgoing echomail messages * Creating 'Outbound Traffic Report' * Build-in help screens for all commands * It does NOT require a specific BBS program, mailer or message editor, as it uses its own control file and environment variable * Writes logfile in FrontDoor, Opus, D'Bridge, Binkley and QuickBBS format Disclaimer, License and Other Important Stuff --------------------------------------------- The following points must be considered, understood and agreed upon BEFORE you use MBUTIL 1.10: * MBUTIL is distributed as ShareWare, and not as FreeWare or Public Domain! * This documentation and the accompanying program is copyrighted material by Gerard van der Land. * Although care has been taken to write and test a program that does what this documentation states, the program is provided as it is, without warranty or guarantee of any kind, either expressed or implied, as to the quality or performance of this program, except that it will occupy disk space. * The author will not be held liable to you or anyone for any direct, indirect, incidental or consequential damages, including any lost profits, lost savings which may result from the use or inability to use this program. Your use of the program constitutes your agreement to this disclaimer and your release of the author from any form of liability or litigation. * You are granted limited license to use MBUTIL 1.10 and you are encouraged to copy and distribute it in an unmodified way as much as you want, as long as you still abide to the conditions mentioned here and no money or other compensation is asked or accepted. 1 System Hardware and Software Requirements ----------------------------------------- * An IBM PC, XT, AT or 100% compatible. * The amount of RAM that should be available very much depends on which command you are using and how big your message base is, but around 300 KB should be enough in most cases. If you run it under DoubleDOS/DesqView, just try. If there's not enough memory available, MBUTIL will certainly let you know! * Microsoft MS-DOS 3.00 / IBM PC-DOS 3.00 or above. MBUTIL was successfully tested under MS-DOS 4.01. * At least "FILES=15" (or more) in your CONFIG.SYS. * A RemoteAccess 0.0x or QuickBBS 2.xx compatible message base. Setup ----- Before you use MBUTIL you should enter your information in the control file MBUTIL.CTL using an ASCII text editor. Explanation of the keywords is given there. Environment Variable -------------------- Usage: SET MBU= [] To be able to run MBUTIL from any directory, you can use the MBU environment variable. If MBUTIL can't find MBUTIL.CTL in the current path, it will try the open the filename set with MBU. This can also be useful for people having more than one message base, who can use SET to select from their batch file which control file should be used. is optional and is the default text color, which is also used if the control file was found in the current path. It is calculated as follows: + (16 * ) Colors: 0 Black 4 Red 1 Blue 5 Magenta 2 Green 6 Yellow 3 Cyan 7 White Add 8 for highlighted foreground colors. is 7 by default (White text on a Black background). Example: SET MBU=C:\RA\MBUTIL.CTL 3 2 Specifying Boards ----------------- If you must specify a , you can use the area tag (if you have specified an AreasFile in the control file), or #. A special fixed area name is "NETMAIL", which will always refer to your NetmailBoard, if defined. Help Screens ------------ If you run 'MBUTIL', you will see the following help screen: +-------------------------------------------------------------------+ | MBUTIL - RemoteAccess/QuickBBS Message Base Utility 1.10 | | Copyright (C) 1990 Gerard van der Land. All rights reserved. | | | | Usage: | | | | MBUTIL [] | | | | Commands: | | | | Export Export netmail messages and replace tearlines | | Import Import netmail messages | | Index Create new index files | | Link Create reply chains and clean subjects | | Move Move messages in one board to another board | | Pack Pack the message base files | | Post Post a file as message in a specified board | | Purge Delete messages according to age or number | | Sort Sort unread or all messages by board/date/time | | Undelete Undelete deleted messages | | | | Run 'MBUTIL ?' to get information about . | | | | C:\RA>_ | | | +-------------------------------------------------------------------+ Example: MBUTIL Purge ? This will give you information about the Purge command and its parameters. If a parameter is missing or invalid, MBUTIL will also show you a help screen. 3 Sort ---- Usage: MBUTIL Sort [-All] [-Backup] [-Pack] The Sort command can be used to sort all unread messages by board number and post date and time. This has two advantages: + When using combined read mode to read new messages for example, you won't see messages from different boards mixed up. + It reduces the chance that you read a reply to a message before you actually read the message that was replied to, which happened pretty often to me and which was actually the reason why I started writing MBUTIL in the first place! But from one thing came another... Reading replies before the original messages is in most cases caused by echo mail processors that don't sort incoming mail packets, but I think that DOESN'T HAVE TO BE the case, since a reply can simply have a shorter or faster PATH than the message that was replied to. MBUTIL checks if there is enough disk space available. After that it scans LASTREAD.BBS to find the highest message that was read by all users. MBUTIL assumes there are actually messages to sort and starts rewriting MSGHDR.BBS and MSGTOIDX.BBS while scanning MSGHDR.BBS to find out where to start sorting. I could have used MSGIDX.BBS for this, but the current way is more reliable and works faster if there actually are messages to sort, which will most likely be the case if you run Sort only each time new messages were tossed in to the message base. MBUTIL will remove all reply chains while sorting, so you should run 'MBUTIL Link' each time you used Sort. The next version might Link while Sorting. This version can sort up to about 9360 messages. You can use '-All' to sort the entire message base, but THIS IS NOT RECOMMENDED FOR BBS SYSTEMS, as IT DOES NOT UPDATE THE LASTREAD.BBS POINTERS (I suggest you delete LASTREAD.BBS if you use this switch). It was only added because some POINTS in the beta-team requested it. Switches: -All Sort entire message base (ignores LASTREAD pointers) -Backup Keep the original MSGHDR.BBS as backup (.BAK) -Pack Remove header records of deleted messages The rest of MBUTIL is actually a clone of TosScan(q) Message Utility (TSUTIL) 1.00, with some additions and improvements here and there. TSUTIL was written by Mats Wallin and Joaquim Homrighausen. I hope they won't be angry with me! ;-) 4 Purge ----- Usage: MBUTIL Purge [-File ] [] [-Convert ] The Purge command can be used to delete messages based on specified per board, like age, received status or maximum number of messages. can be a command line or a text with lines in the following format: [] [-All] [-Days ] [-Rcvd] [-Msgs ] can also be "RemoteAccess" or "TosScan", which will use their files, if found in the current path. If you add '-Convert ', you can convert TosScan or RA information to a text file. If you use '-Msgs -Days -Rcvd -Convert ', your AreasFile will be converted to purge file format. You may put comments after a ; in a text . If you use '-File -Days -Msgs ', and will apply to all boards for which the number of Days/Msgs was not defined (zero) in . If you use '-File -Days -Msgs ', and will override the settings in . If you use '[-File ] [] -Rcvd', received messages will be deleted in or all boards. If you use '-File RemoteAccess [...]', it will delete received messages in all boards where the "Days After Received Messages are Killed" value in RACONFIG is non-zero. If you want to, you can still retrieve deleted messages with 'MBUTIL Undelete', described later. Switches: -All Delete all messages (won't be used as default) -Days Delete messages older than -Rcvd Delete all received messages -Msgs Delete all except the last messages -Convert Convert AreasFile, MESSAGES.RA or AREAFILE.FD Examples: ; Delete messages older than 2 weeks and received messages on board 10. MBUTIL Purge #10 -Days 14 -Rcvd ; Delete messages in SYSOPS.028 according to information in MBUTIL.PUR MBUTIL Purge -File MBUTIL.PUR SYSOPS.028 5 Pack ---- Usage: MBUTIL Pack [-Backup] [-Force] [-Overwrite] [-Renumber] [-Delete] [-Recover] The Pack command can be used to pack the message base files by removing deleted messages. Pack will by default rewrite the message base file and delete the old files if the message base was packed successfully. If you use '-Backup', it will keep MSGHDR.BAK and MSGTXT.BAK. Note that I don't keep a backup of the index files. If you should ever need to use your backup message base, just run 'MBUTIL Index' to create new index files. By using '-Overwrite' you can choose to overwrite the existing files if there's not enough disk space to create new files. I personally always use and recommend using '-Force' to overwrite the existing message base files, which has the following advantages and disadvantages: + It requires no free disk space at all, which is actually the reason why I always HAVE to use overwrite mode. :-) + It is much faster (especially since MBUTIL won't rewrite text blocks that don't need to be moved) + It causes less disk fragmentation, and thus prevents message base access from slowing down. - It is a little bit more dangerous, because if MBUTIL is interrupted while overwriting, it will probably damage your message base beyond repair. 'MBUTIL Index -Delete -Renumber' is the best you can do when this should happen. MBUTIL will temporary ignore the Ctrl-Break and Ctrl-C keys when it is in overwrite-mode so you can't accidentally interrupt it. When MBUTIL has renumbered the messages, it will update the LASTREAD pointers in LASTREAD.BBS. If USERS.BBS exists it will also update the 'highest read' pointers in the user records and reformat LASTREAD.BBS if necessary. I recommend using '-Renumber' regularly. MBUTIL will automagically renumber if the highest message number is higher than what you defined using the 'AutoRenumber' keyword in the control file. See 'MBUTIL Index' on the next page for details on using '-Delete' and '-Recover'. Switches: -Backup Keep the original MSGHDR.BBS and MSGTXT.BBS as .BAK -Force Force overwriting of the existing message base files -Overwrite Overwrite existing files only if short of disk space -Renumber Renumber messages and update LASTREAD pointers -Delete Delete messages crosslinked in MSGTXT.BBS -Recover Move messages in invalid boards to the RecoveryBoard 6 Index ----- Usage: MBUTIL Index [-Delete] [-Recover] [-Renumber] The Index command can be used to create the following index files: MSGIDX.BBS This file contains the message and board numbers of each parallel records in MSGHDR.BBS. It is used to quickly find a message by message number or to find the previous/next message on a specific board or in the entire message base. MSGTOIDX.BBS This file contains the name of the addressees of the messages in the parallel record in MSGHDR.BBS. It contains "* Received *" if a message is already received and "* Deleted *" if a message is already deleted. It is used to quickly check the entire message base for personal mail. MSGINFO.BBS This file contains the total number of messages in the entire message base, the highest and lowest message numbers used and the number of messages per board. It is used to give the BBS program, message editor or echo mail processor this information without having to scan MSGHDR.BBS. Note that these index files are also created by the Pack, Purge and Undelete commands. If Sort sorted message it will also create new index files. You can optionally delete crosslinked messages (two or more messages claiming partially or entirely the same block in MSGTXT.BBS) or messages pointing to a non-existing block (outside MSGTXT.BBS). You can also choose to move messages in invalid boards (0 or > 200) to the RecoveryBoard. If you haven't defined one, these messages will be deleted if you use the -Delete or -Recover switches. Please note that -Delete and -Recover don't update the reply chains. If the message numbers really get out of order and/or Pack is unable to renumber the messages (for instance if there are negative message numbers or you don't have enough memory available) you can use the -Renumber switch to fix this. Please note that this will *NOT* update the LASTREAD pointers. Only use this switch in extreme emergencies. If you don't have very much memory you should run 'MBUTIL Pack -Renumber' regularly or choose a low AutoRenumber value to prevent needing this. Switches: -Delete Delete messages crosslinked in MSGTXT.BBS -Recover Move messages in invalid boards to the RecoveryBoard -Renumber Renumber all message numbers in MSGHDR.BBS 7 Link ---- Usage: MBUTIL Link [-Clean] [-Remove] The Link command can be used to link messages with the same subject in a reply chain that can be walked through with the message editor or the BBS program. MBUTIL will always create new reply chains for all boards rather than updating any existing reply chains. Deleted messages will not be linked. MBUTIL links messages using a CRC-32 on the message subjects, ignoring case and "Re:" or "(R)" junk. Future versions of MBUTIL will use the MSGID and REPLY kludges and/or sort the reply chains by post date. With the -Remove switch MBUTIL will only remove all existing reply chains. With the -Clean switch MBUTIL will remove "Re:" and "(R)" junk from the subject texts. I recommend running 'Link -Clean' each time messages were tossed into the message base. Switches: -Clean Remove "Re:" and "(R)" from subjects -Remove Remove all existing reply chains Move ---- Usage: MBUTIL Move The Move command can be used to move all messages in one board to another board. The messages are not physically moved, only their board number is changed. They keep the same message numbers and reply chains. Deleted messages will not be moved. 8 Export ------ Usage: MBUTIL Export [-Crash] [-Direct] [-Keep] [-ReTear] [-Report] or: MBUTIL Export [-Delete] The Export command is the opposite of the Import command and can be used to export messages from RemoteAccess message boards to FidoNet .MSG standard messages. It will by default export netmail messages from all boards to the NetmailPath if defined. If netmail was entered using RemoteAccess, those programs will increment the Pending field in the user record. When MBUTIL has exported such messages, it will decrement both Pending and Credit fields in the user record (if found). Netmail will NOT be exported if it is destined to your main address or one of your Aka's. When export is used to export netmail, it will also turn off invalid 'Outgoing EchoMail' bits of messages that don't have the Local bit set, which fixes the problem that Qecho exports messages that you replied to with FrontDoor 1.99c. MBUTIL will also create a file TRAFFIC.MBU, that contains the number of outbound messages from each board. Use '-Report' to produce the report TRAFFIC.RPT. When you use the -ReTear switch, MBUTIL will search outgoing echomail messages for up to five specified tearlines and replace them with the matching ones, comparable to ReTear by Cary Howell. If you specify , you can use MBUTIL to Export all messages in an RemoteAccess board to .MSG format. Don't use this to export netmail! Switches: -All Send all messages to another zone direct, not via the zonegate (myzone:myzone/destzone) -Crash Send only crash messages to another zone direct -Keep Keep the message in the NetmailPath after they have been sent. Normally to are killed after they have been sent. -Delete Delete messages in the RemoteAccess board after they have been exported. This is always done for netmail with the Kill/Sent bit set, even if you use -Keep. -ReTear Replace the tearlines of Outgoing Echomail messages 9 Import ------ Usage: MBUTIL Import [-Users] or: MBUTIL Import [-Delete] The Import command is the opposite of the Export command and can be used to import messages from FidoNet standard .MSG message files to RemoteAccess message boards. It will by default import messages from the NetmailPath to the NetmailBoard if defined. It only imports netmail messages that are destined to your main address or one of your Aka's and if the name of the addressee is one of the users listed in the control file or found in the user file. If a message is for one of the users listed in the control file, MBUTIL will still search the user file (if any) for the username, and replace it exactly as it was written there. If you use the -User switch, MBUTIL will NOT import messages addressed to users listed in the control file. If you specify , you can use MBUTIL to Import an entire directory of .MSG format echomail messages, for example to import messages from you Dupe path to a RemoteAccess board. Don't use this to import netmail! Switches: -Delete Delete the .MSG files after they have been imported in they have been imported in the RemoteAccess. This is always done for netmail. -Users Don't import messages to any of the users defined in the control file Examples: ; Import netmail MBUTIL Import ; Import all .MSG files in C:\FD\BADECHO to the area "NEWECHO" and ; then delete the .MSG files MBUTIL Import C:\FD\BADECHO NEWECHO -Delete 10 Post ---- Usage: MBUTIL Post [-From ] [-To ] [-Subject ] The Post command is similar to the Import command and can be used to create a message with the contents of a specified file in a specified board. The messages will have the 'Outgoing Echomail' bit set, so they will be exported if you post them in an echomail board. Post can't post files larger than 32 KB. If , or contains spaces, enclose it in double quotes. Defaults: -From First UserName defined in control file -To All -Subject Example: MBUTIL Post C:\MBU_REL.TXT 37 -From "Gerard van.der.Land" -To "Everyone" -Subject "Message Base Utility 1.10 release" One special option not listed on the help screen is the -Test switch. If you use it, the Post command will have the following format: MBUTIL Post -Test [...] MBUTIL will post test messages in the specified . Those message will NOT be exported. I used this switch for testing purposes and decided to keep in the release version. Undelete -------- Usage: MBUTIL Undelete [] The Undelete command can be used to undelete all deleted messages in all boards or . When messages are deleted, they are marked as deleted and not physically deleted or overwritten, which means that you can Undelete all deleted messages, as long as the message base file haven't been packed yet. 11 AreasFile Example ----------------- This is an example for an AreasFile. This file, usually called AREAS.BBS, is used by some echomail processors, like Qecho and ZMail. /--------------------------------------------------------------------\ Default origin line ! Sysop name 1 RA_SUPPORT [net/nodes...] 2 RA_SUP.028 [net/nodes...] 3 LOCAL \--------------------------------------------------------------------/ In this example "RA_SUPPORT" would be the same as "#1". As you can see, you can add local areas to make it easier to specify those boards. Purge File Example ------------------ This is an example for a Purge File. /--------------------------------------------------------------------\ ; MBUTIL 1.10 Purge File ; ; Example ; ; 0 Days or 0 Msgs = unlimited ; ;Board # of Days # of Msgs Rcvd Comment (optional) ;----- --------- ----------- ----- -------------------------------- -Days 21 -Msgs 100 ; Default values #001 -Days 14 -Msgs 200 ; Int. RemoteAccess Support Area #002 -Days 21 -Msgs 50 ; Nat. RemoteAccess Support Area #003 -Days 28 -Msgs 150 -Rcvd ; Local sysop board ; ; It can also be like this: ; RA_SUPPORT -Days 15 -Msgs 300 ; RemoteAccess Support Area NETMAIL -Days 14 -Msgs 200 -Rcvd ; This is the NetmailBoard LOCAL -Days 10 -Msgs 75 -Rcvd ; Local board \--------------------------------------------------------------------/ You can also create a Purge File with the -Convert switch: To convert MESSAGES.RA to a text file: MBUTIL Purge -File RemoteAccess [...] -Convert PURGE.RA To convert AREAFILE>FD to a text file: MBUTIL Purge -File TosScan [...] -Convert PURGE.TS To convert your AreasFile to a text file: MBUTIL Purge [...] -Convert PURGE.MBU You can also specify default values: MBUTIL Purge -Days 14 -Msgs 50 -Rcvd -Convert PURGE.MBU 12 Batch File Example ------------------ /--------------------------------------------------------------------\ :Clean rem Do your daily/nightly maintenance here. rem The Purge command depends on what software you are using. rem In a RemoteAccess environment you can use: CD \RA MBUTIL Purge -File RemoteAccess rem In a FrontDoor+TosScan environment you can use: CD \FD MBUTIL Purge -File TosScan rem This works in all environments: MBUTIL Purge -File MBUTIL.PUR rem This command deletes invalid messages, recovers messages in rem invalid boards and packs and renumbers the message base by rem overwriting the existing message base files MBUTIL Pack -Force -Renumber -Delete -Recover goto Start :Unpack rem Toss incoming echomail, import netmail, sort unread messages rem and create reply chains MBUTIL Import MBUTIL Sort -Pack MBUTIL Link -Clean goto Start :Net rem Only export NetMail MBUTIL Export goto Start :Net&Echo rem It is not necessary that there is NetMail that needs to be rem exported, but we run Export anyway for the ReTear option. rem We are sure that EchoMail needs to be exported. rem Just replace the "goto Echo" with "goto Net&Echo" MBUTIL Export -ReTear goto Start \--------------------------------------------------------------------/ 13 History/Future -------------- Until now I spent quit some hours writing, testing and documenting MBUTIL and I will keep on doing that. There are also things that can and will be done to make MBUTIL even a little bit smaller and faster. There is still some redundant code that could be made more compact and efficient. MBUTIL is also using quit a lot of memory at the moment, which can be made less. Further I'm working on my own echomail processor and a full screen personal mail check. Registration ------------ As long as you don't register, MBUTIL will remind you of that on the screen and in the logfile. MBUTIL is NOT crippled in any way. If you use this program for a period of time and you decide you like it, you can get a personal RegistrationKey by sending: * $10US - $99US (Whatever the program is worth to you, may also be an equivalent amount in a freely convertible currency). * The name you want to have defined after the "RegisteredTo" keyword. * Any questions, comments, suggestions and bug-reports, and ofcourse your FidoNet (or else snail mail) address, to: Gerard van der Land Cannenburg 23 3772 BH Barneveld The Netherlands FidoNet 2:283/1.5 and 2:283/108.1 Users in The Netherlands can transfer their registration to PostBank giro-account 5.245.162 in The Netherlands. For usres outside The Netherlands there are a few other possibilities to send your registration: * Cash money in a registered letter. * Sending a EuroCheque. * International money order. * Personal check. If you absolutely want to do this, the minimum registration fee is $17US, because of the outrageous conversion cost the banks charge when cashing a check. You will receive your key per crashmail (or snailmail if you are not in FidoNet) as soon as possible after I received the money or money order form or check. Don't forget to register! 14 Technical Notes --------------- * MBUTIL was written using Turbo C 2.00 and Turbo Assembler 1.01. * It writes directly to video RAM or DesqView/DoubleDOS virtual screen buffer. * It can't be used while an other program is also using the message base at the same time (multitasking environments). It does not support record locking yet, which might change in the near future. * Import and Export are fully zone and point aware. * Export creates FTS-0001 rev 12 compatible .MSG files. * Import can parse Fido and SEAdog style .MSG dates. * Link is based on code written by Folkert Wijnstra. * You can use LZEXE 0.91 by Fabrice Bellard to reduce the size of MBUTIL.EXE down to 32 KB. * TRAFFIC.MBU has the follow format: struct TRAFFIC.MBU { unsigned int Outbound[200]; /* # of outbound messages per board */ unsigned char Month, Day, Year, Hour, Minute; /* file created */ unsigned long TotalCost; /* total netmail cost */ } Traffic; Many Thanks To... ----------------- ... Eric Lotgerink, Onno Hagers, Perry de Ronde, Reinier de Groot, Adrie van Lith, Folkert Wijnstra, Gerard Sittrop, Rudi van Lishout, Richard B. van Dijk, Willem Blink, John Veldhuis, Sander Davidson, Sander Eek, Ari Sadarjoen, Peter Smink, Jan Vroonhof, Arjen Lentz and all who I forgot... All brand and product names are Copyrighted (C) material, Trademarks or Registered Trademarks (R) of their respective holders: BinkleyTerm Bit Bucket Software D'Bridge Chris Irwin DesqView Quarterdeck Systems DoubleDOS SoftLogic Systems FrontDoor, TosScan Joaquim H. Homrighausen IBM International Business Machines Corporation MBUTIL Gerard van der Land Microsoft, MS-DOS Microsoft Corporation Opus Wynn Wagner III QuickBBS, Qecho Ultimate Software Technologies, Inc. RemoteAccess Continental Software, Inc. ReTear Cary Howell TSUTIL Mats Wallin and Joaquim H. Homrighausen Turbo C, Turbo Assembler Borland International, Inc. 15 Other Files I Released Recently ------------------------------- * FD199CNG.ZIP (84 KB): The FrontDoor 1.99c Norton USER GUIDE database This archive contains FD199C.NG, which needs NG.EXE, the resident Norton Guides program. It enables you to consult the FrontDoor manual any time you want by pressing a hot key. It uses menus with the same conveniently arranged index as FRODO.DOC does. * FS_100.ZIP (7 KB): Folder sort 1.00 A program that enables you to organize your FrontDoor FOLDER.SYS by manually moving the folders to the positions where you want them. You can also sort by title, delete folders or duplicate them to save time with adding new ones. * USA_100.ZIP (21 KB): USERS.BBS Statistics Analyzer 1.00 A statistics program for RemoteAccess and QuickBBS 2.xx. It is very fast, small, and fully configurable. It can produce the following QuickBest look-alike surveys: + Best Downloaders (KB) + Best Uploaders (KB) + Best Downloaders (Times) + Best Uploaders (Times) + Best Callers + Best Message writers + Last Callers + Best active Areas + File/Message Base Report + Security Report with up to 7 Security Levels + Flag Report with Setting Flags and up to 7 Authorization Flags The latest version will always be requestable as "USA" and "MBUTIL" from: 2:282/220 (V22b), RAccess The Arena, Stiens Holland *==================* Pfff... End Of Documentation *==================* 16

---

E-Mail Fredric L. Rice / The Skeptic Tank