From Newsbin

Quick Links: Version 5 Documentation | Version 6.60 Documentation | Newsbin Home | Latest Newsbin Release | Glossary | Newsbin Forums

Documentation in progress...

Contents 1 Downloading Files 1.1 Selecting Files for Download 1.1.1 Post Lists 1.1.2 Using the Filter bar 1.1.3 Using the Search facility 1.1.4 Duplicate files 1.1.5 Minimizing Memory Usage while Selecting files 1.2 Marking Headers as Read/Deleted 1.3 Text Files 1.4 Binary Files 1.4.1 Single Part Posts 1.4.2 Multi-part Posts 1.4.3 Incomplete Posts 1.4.4 RAR Sets 1.4.5 Split Files 1.5 Where are my Downloaded Files?

Downloading Files

After you have downloaded headers, you can view the headers in a Post List by selecting a group and hit ctrl-L (load spool) or right-click and select Show Posts. Once the post list is populated, you can select files to download. The process of selecting files to download involves adding posts to the Download List. Do this by either clicking in the box to the left of the subject, or right-click the post, or a selection of posts, and select "Add to Download List".

As you select posts for download, you will see a line appear in the Download tab which is in the bottom half of the NewsBin window by default. By default, each selected post is added to the bottom of the list. Items are downloaded in the order they are displayed in the Download list, so if you want to prioritise any items move them up the list.

Tip If you hold down the SHIFT key when you select a post for download, it will be added to the top of the list rather than the bottom, and downloading will start as soon as a connection becomes free after having completed the chunk it is working on.

Selecting Files for Download

Post Lists

When you decide that you would like to view previously downloaded headers you load them into what in Newsbin is known as Post list. It is from this Post list that you view what is available, search for items of particular interest and select the items that you want to download. To load headers into a Post List you use either the Show Posts or Show All Posts options. The difference is that the Show Posts limits the headers to those that are newer than the Maximum Post Age setting, while Show All Posts ignores this setting.

Tip If you normally only want to look at the last few days of headers then set the Maximum Post Age (MPA) setting to something like 5 days. This will vastly speed up loading groups. On the rare occasions when you want a longer period you can use the Show All Posts facility; or change the MPA value and continue to use Show Posts; or finally use the search facility to search the headers on disk.

To help remove unnecessary detail in a Post list Newsbin will try and aggregate headers that refer to a single file into a single Post list line. This means that you will have less entries in a Post list that the number of headers whose information was used to build the list. If you have selected the "Compact" option at the head of a Post list then a single line can sometimes refer to thousands of headers. You can always see the scale of this aggregation effect by right-clicking an entry in the list and selecting the "Properties" option to see what headers are associated with that Post list line.

Normally you will probably load a single group into a Post list, but it is also possible to load multiple groups into one list - and in that case any cross-posted entries will only appear once. There are two ways of achieving this:

• Organize multiple groups in the Group list into a "Group of Groups" (GoG), and select the GoG as the item to be downloaded. All groups associated with that GoG will then be loaded.
• Highlight several groups and select the "Show Posts" option. These will then all be loaded into the same Post list.

The one thing you have to be careful with loading the headers for multiple groups in this way is that you must sufficient RAM to handle all the headers involved.

You can also select whether when you open a Post list you are restricted to a single list with the new one replacing the previous list, or alternatively every time you open a list you get a new tab created. Some people prefer the simplicity of the single Post list approach while others like the flexibility of having multiple Post lists. This behavior is controlled by the setting of the Multi-tab mode under Options->Switches.

You can select to sort any list by clicking on the header of the column you want to sort on. Clicking again on the same header will result in the sort list being inverted. When you close a list, the sort order current at the time is remembered and becomes the initial sort order for when you next open that list.

Using the Filter bar

The filter bar is present at the top of Post lists and looks like this:

The purpose of the Filter Bar is to help with selecting the posts that might be of interest from the Post list with which it is associated. Whether the filter bar is active or not is controlled by whether you ahve ticked the "Enable Filters" box at the top of the bar.

A big plus point on the Filter bar is that it is works purely within memory so is very fast. This makes it easy to play around with different settings to find items of interest. The disadvantage is that you have to first load the group(s) into a Post list, and if the group is very large you may not have enough RAM to achieve this.

For more information on each of the fields within the Filter bar refer to the ??

Using the Search facility

The search facility differs from the filter facility mentioned above in that instead of looking at the headers loaded into memory in a Post list, it instead looks though the headers stored on disk for the selected group(s). This is much slower that using the Filter bar, but it means that you do not have to first load the Post list, and possibly more important you can handle very large groups that you may not have sufficient RAM to load into memory.

To use the Search facility you open up the Search tab (or press the Search button the toolbar which has the same effect). At the top of the Search tab is the area where you enter he search criteria. It looks like this:

As you can see it is very similar to the Filter bar used on Post list, with slight changes due to its different purpose.

Duplicate files

Newsbin has a feature known as the Signature Cache to stop you downloading duplicate files. It does this by storing checksums of the first part of already downloaded files, so when a new file comes along it can download the first part; generate this checksum; and check it it against its stored database rejecting any duplicates.

Most of this time this is exactly what you want, but occasionally for some reason you may want to download a file again. In this case you can either:

• Disable the Signature Cache (not recommended) so that the check is suppressed.
• Bypass the check. The option can be called Force Download or Bypass filters depending on which menu it is on, but it will always have the keyboard shortcut of CTRL-Y (or possibly SHIFT-CTRL-Y if you want to re-queue to the top of the Download queue). This option is available in most lists.

Minimizing Memory Usage while Selecting files

One needs to be aware of the fact that if you are trying to handle some of the largest groups on Usenet then one needs to be aware of the techniques that can be used to minimize memory usage within Newsbin. Ask Newsbin to do too much, and it will bring even the most powerful machine to its knees.

Techniques to be aware off are:

• Separate Downloading Headers from Viewing them: Newsbin 5 supports downloading headers to be done asynchronously to viewing them. If you do not have a post list open when downloading headers for the group then they are simply written to disk keeping memory usage down. If a Post List is open then copies have to be kept in memory which can make a dramatic difference in memory usage.
• Use Maximum Post Age (MPA) setting:: The MPA setting is used to limit how many days worth of headers are loaded into memory when you open a Post List. You can set the default value under Options->Setup, and group specific overrides under the properties of individual groups. If you are only interested in recent posts there is no point in loading older posts, and keeping the number down makes Newsbin far less memory hungry and far more responsive. Note that the difference between "Show Posts" and "Show All Posts" on the groups right-click menu is that Show Posts respects the MPA value, while Show All Posts ignores it.
• Avoid Multiple Post lists: If you have the Multi-tab Mode option set under Options->Switches then Newsbin allows each group to be loaded into its own Post list. While this can be very convenient when handling small or medium sized groups it should be avoided when handling the very largest groups.
• Avoid multi-group Post lists: Newsbin allows you to load multiple groups into a single Post list. this can be done by organizing the groups into a "Group of Groups" (GoG) and selecting the GoG when opening a Post list. Avoid using this technique on the largest groups.
• Use the Search facility: The Search facility can be used to search groups that are too large to handle in the RAM that you have on your PC. The Search facility reads through the spool files on disk, and only loads those headers that meet your search critieria. The penalty is that this is slower than using the Filter Bar on a Post list, but it does allow very large groups to be searched on machines that do not have enough RAm to load the group into a Post list. There is also an option on the Search tab to limit the number of days worth of headers to be examined - using this will speed up searches just as using the MAP value does when loading Post lists.

Marking Headers as Read/Deleted

When headers are downloaded by Newsbin they are flagged with a New status when they are written to disk. It is quite normal to want to get this status cleared when you have looked at the headers and decided what to do with them as this then allows you to exploit the Hide Old checkbox on the Filter Bar at the top of Post lists so that you only see posts that are new (or you have not yet dealt with).

the New status of a Post can be cleared in several ways:

• Adding the file to the Download queue
• Marking the file as Read. This can be done by either using the DEL key, or one of the options from the right-click menu. If you have selected a range of posts then the action will apply to all the selected posts. The update of the headers on disk runs as background task so it may take a short while to complete so that if you mark posts as Read and then immediately try to reload the group it may not have completed the task so some of them still show as new.
• Get Newsbin to automatically clear the new status from the disk copy of the headers once it is loaded into a Post list. To do this you need to set the Auto-Mark Old setting under Options->Switches. Newsbin actually clears the new status from the disk headers as soon as it has loaded them into the Post list rather than at the point the Post list is closed.

Note that marking a post as Read does not stop it being loaded into memory - it is still available for viewing if you untick the Hide Old checkbox on the filter bar at the top of the Post List. It also means it is still using memory when the Post list is loaded which could be a factor in very large groups. Post can manually be set back to a New status if you accidentally mark the wrong ones as read via the option on the right-click menu.

The other facility is to mark a post as Deleted using either SHIFT-DEL or an option from the right-click menu. You would do this if you were absolutely certain that you would never want to see that post again. This does not physically remove it from the local hard disk, but it does stop the Post from ever being loaded into memory again and consuming resources. It will be physically removed from the hard disk if any of the following occur:

• You do a header download and the server reports that the headers concerned are no longer on the server.
• You do a manual Purge to Global MRA' and the post is older than the MRA value.
• The post reaches an age limit (currently 120 days) after which it is purged from disk regardless.

Text Files

NewsBin was written to download binary files rather than text files. A basic facility has been implemented to read text files - you are likely to want this to read items like .NFO information files associated with binary files.

Text posts can normally be identified by the fact that they tend to be very small posts (<3K). If you want to read these from within Newsbin you use the Read Post Body feature. This is used as follows:

• Select the post and hit ctrl-r or right-click and select Read Post Body.
• The post will be loaded into the text file viewer that you have set as the default viewer for .txt files at the Windows level. Note this did not happen in the early releases of Newsbin 5 prior to the 5.05 release where you always had to go to the Read Post Body tab as described in the next step.
• The post will also have been added to the Read Posts tab (View->Read Posts to activate if not already visible). You can go there at any time if you want to re-read any recent text posts on which you have used the "Read Post Body" feature. You may have to hit the Reload button on the tab the first time you bring it up.

Binary Files

This is NewsBin's strength, downloading binary files. There are single part and multi-part files, and different encoding schemes. NewsBin hides the complexity of usenet by automatically downloading and decoding individual posts that make up a complete file.

To select a file for download and send it to the Download list you can do any of the following:

• Click the box to the left of the subject.
• Double-Click the post entry
• Highlight the post, right click and select "Add to Download List"
• Highlight the post, and then use the CTRL-E keyboard shortcut

You can also highlight multiple posts using standard windows multi-select methods such as using SHIFT or CTRL keys when highlighting the posts, and then add them all in one go to the Download list using the above methods.

Single Part Posts

Single part files are smaller files that can be downloaded in a single chunk by a single connection. The status column will not appear to be segmented, and the size column is usually measured in KB because of article size limits on usenet posts.

Multi-part Posts

For multi-part posts, the Progress Bar is segmented to represent each part that needs to be downloaded. As each part (or chunk) is downloaded, the color of the server being used to download it is displayed in the lower part and green in the upper part. If the download of that chunk completes successfully, the segment will all show as the color of the server that seceded, while if it fails the segment is turned red.

You should avoid using any of the system colors for your server.

Incomplete Posts

You can create files from incomplete posts by adding them to the download list from the post list. NewsBin will try to download as much as it can. It will stay in the download list unless you have "Assemble Incompletes" selected under Options/Advanced. This only works if there are no errors downloading the parts that NewsBin thinks exists. If there are errors, you can manually assemble the incomplete by right-clicking in the download list and select "Assemble Incompletes" or "Assemble Incompletes and Remove" if you want to remove it from the download list after it is assembled.

Assembled incomplete files can be completed by PAR2 files.

RAR Sets

RARs are the archives and PARs are there to help verify and repair after downloaded. These Rar and Par files will be compacted together to appear as one large file by default until you choose to uncompact or expand the set of files to to add or see the list of individual files.

Split Files

Split files created with Master Splitter or HJ Split will have a numbered extension and will be joined back together in Newsbin if they were posted with a par2 file that was created using the original file.

Where are my Downloaded Files?

NewsBin has several settings to allow you to specify where your files should be stored.

Note that if you change any of these settings they only apply to new files you add to the download list - they are not applied retrospectively to those files that are already queued.

• Global Download Path
  The main setting is under Options/Setup using the "Download Path" setting. By default, all files will be stored in this folder, or in a sub-folder relative to this path.

• Group Download Path
  You can configure a different Download Path for each newsgroup or Group of Groups (goG) by setting the Download Path under the Group Properties. Get there by right-clicking on the newsgroup in the Groups tab and select "Properties". If set the GoG value over-rides the global value, and the individual newsgroup level over-rides both the GoG and global levels.this will over-ride the Global Download Path.

• Manual Download Path
  You can also manually specify a download folder when you add a file to the Download list from the Post list using the "Download to Folder" option.

The other point that is commonly required is to get files stored in sub-folders relative to the Download path. There are several settings that can affect this:

• Folder Mode
  This setting (under Options->Filename Options) allows you to specify that the files should be put into a folder named after the group relating to the item selected. Note that if you want the same behaviour for files that are selected from NZB files then you also need to set the option under Options->NZB Files.
• MP3 Mode
  This setting (under Options->Filename Options) is specific to MP3 files and means that you want the files to be put into a sub-folder whose name is derived from the ID3V2 tags embedded in the file. This does rely, however, on there being good values set for these tags, and very often this is not the case. It also has the disadvantage that any associated files (e.g. covers or playlists) that are not mp3 files will not end up in this folder.
• NZB Folder Mode
  This setting (under Options->NZB Files) is specific to NZB files and specfiys that the file downloaded should be put into a folder whose name is derived from the NZB file itself. You can also specify whether the prefix should be stripped off the name used to create the folder.

There are various levels of combination of the above setting that can be used. If you are not sure how they interact, you can always see what destination will be used for a specific file in the Download list by looking in the Download path column.

If the files have already been downloaded, then there is a column in the Files tab that has the path where the files are saved. You can also right-click on a file in this list and select "Open Folder" to get to the folder where the files are.