From Newsbin

Version 6 Help Page | Version 6 Bug Tracking | Beta Notes | V6 Beta Forum | Newsbin Beta Page

Contents 1 Downloading Files 1.1 Selecting Files for Download 1.1.1 Post Lists 1.1.2 Using the Filter bar 1.1.3 Using the Search Tab 1.1.3.1 Internet Search 1.1.3.2 Local Mode 1.1.3.3 Profile Mode 1.1.4 Duplicate files 1.1.5 Minimizing Memory Usage while Selecting files 1.2 Where are my Downloaded Files? 1.3 Marking Headers as Old/Deleted 1.4 Text Files 1.5 Binary Files 1.5.1 Single Part Posts 1.5.2 Multi-part Posts 1.5.3 Incomplete Posts 1.5.4 RAR Sets 1.5.5 Split Files

Downloading Files

After you have downloaded headers, you can view the headers in a Post List by selecting a group and hit ctrl-L (load spool) or right-click and select Show Posts. Once the post list is populated, you can select files to download. The process of selecting files to download involves adding posts to the Download List. Do this by either clicking in the box to the left of the subject, or right-click the post, or a selection of posts, and select "Add to Download List".

As you select posts for download, you will see a line appear in the Download tab which is in the bottom half of the NewsBin window by default. By default, each selected post is added to the bottom of the list. Items are downloaded in the order they are displayed in the Download list, so if you want to prioritize any items move them up the list.

Tip If you hold down the SHIFT key when you select a post for download, it will be added to the top of the list rather than the bottom, and downloading will start as soon as a connection becomes free after having completed the chunk it is working on.

Selecting Files for Download

Post Lists

When you decide that you would like to view previously downloaded headers you load them into what in Newsbin is known as Post list. It is from this Post list that you view what is available, search for items of particular interest and select the items that you want to download. To load headers into a Post List you can use either the Show Posts or Show All Posts options from the right-click menu. The difference is that the Show Posts limits the headers to those that are newer than the Display Age setting, while Show All Posts ignores this setting. You can also double-click a group and this will combine a Show Posts to load the posts already downloaded from the disk with a Download Latest to get any new posts from the server.

Tip If you normally only want to look at the last few days of headers then set the Display Age setting to something like 5 days. This will vastly speed up loading groups. On the rare occasions when you want a longer period you can use the Show All Posts facility; or change the Display Age value and continue to use Show Posts; or finally use the search facility to search the headers on disk.

To help remove unnecessary detail in a Post list Newsbin will try and aggregate headers that refer to a single file into a single Post list line. This means that you will have less entries in a Post list that the number of headers whose information was used to build the list. If you have selected the "Compact" option at the head of a Post list then a single line can sometimes refer to thousands of headers. You can always see the scale of this aggregation effect by right-clicking an entry in the list and selecting the "Properties" option to see what headers are associated with that Post list line.

Normally you will probably load a single group into a Post list, but it is also possible to load multiple groups into one list - and in that case any cross-posted entries will only appear once. There are two ways of achieving this:

• Organize multiple groups in the Group list into a "Group of Groups" (GoG), and select the GoG as the item to be downloaded. All groups associated with that GoG will then be loaded.
• Highlight several groups and select the "Show Posts" option. These will then all be loaded into the same Post list.

The one thing you have to be careful with loading the headers for multiple groups in this way is that you must have sufficient RAM to handle all the headers involved.

You can also select whether when you open a Post list you are restricted to a single list with the new one replacing the previous list, or alternatively every time you open a list you get a new tab created. Some people prefer the simplicity of the single Post list approach while others like the flexibility of having multiple Post lists. This behavior is controlled by the setting of the Multi-tab mode under Options->Switches.

You can select to sort any list by clicking on the header of the column you want to sort on. Clicking again on the same header will result in the sort list being inverted. When you close a list, the sort order current at the time is remembered and becomes the initial sort order for when you next open that list.

Using the Filter bar

The filter bar is present at the top of Post lists and looks like this:

The purpose of the Filter Bar is to help with selecting the posts that might be of interest from the Post list with which it is associated. Whether the filter bar is active or not is controlled by whether you have ticked the "Enable Filters" box at the top of the bar.

The Filter bar applies to posts resident in memory so it is very fast. This makes it easy to play around with different settings to find items of interest. See Filters.

Using the Search Tab

There are three Search modes, Internet, Local, and Profile.

To open the Search tab, hit the F3 key to switch to the tab or press the Search button on the toolbar. At the top of the Search tab is the area where you select the Search Mode and enter the search criteria. It looks like this:

As you can see it is very similar to the Filter bar used on Post list, with slight changes due to its different purpose.

Internet Search

The default Search Mode is our optional Internet Search service. In this mode, NewsBin send the keyword you enter to our database over the Internet and a list of matching posts is returned and will fill the Search tab. This is an optional service we provide for a fee. New users receive 10 free searches to try it out. More information about NewsBin Internet Search is here and in Searching and Filtering.

Local Mode

The Search facility in Local mode differs from the filter facility mentioned above in that instead of looking at the headers loaded into memory in a Post list, it instead looks though the headers stored on disk for the selected group(s). This is much slower that using the Filter bar, but it means that you do not have to first load the Post list, and possibly more important you can handle very large groups that you may not have sufficient RAM to load into memory.

Profile Mode

This mode allows you to apply a filter profile to the current list of Search results. You would first perform a search in either Internet or Local mode then, to further narrow down your search results, you can select "Profile" and then the available Filter Profiles will be shown in the dropdown box where you entered your original Search keyword.

Duplicate files

Newsbin has a feature to help keep from downloading duplicate files. It does this by storing checksums of the first part of already downloaded files, so when a new file comes along it can download the first part; generate this checksum; and check it it against its stored database rejecting any duplicates.

Most of this time this is exactly what you want, but occasionally for some reason you may want to download a file again. In this case you can either:

• Disable the Duplicate Detector (not recommended) so that the check is suppressed.
• Bypass the check. The option is called Add to Download List - Bypass filters or use the keyboard shortcut of CTRL-Y (or possibly SHIFT-CTRL-Y if you want to re-queue to the top of the Download queue). This option is available in most lists.

Minimizing Memory Usage while Selecting files

One needs to be aware of the fact that if you are trying to handle some of the largest groups on Usenet then one needs to be aware of the techniques that can be used to minimize memory usage within Newsbin. Ask Newsbin to do too much, and it will bring even the most powerful machine to its knees.

Techniques to be aware off are:

• Separate Downloading Headers from Viewing them: Newsbin 5 supports downloading headers to be done asynchronously to viewing them. If you do not have a post list open when downloading headers for the group then they are simply written to disk keeping memory usage down. If a Post List is open then copies have to be kept in memory which can make a dramatic difference in memory usage.
• Use Dispaly Age setting:: The Display Age setting is used to limit how many days worth of headers are loaded into memory when you open a Post List. You can set the default value under Options->Setup, and group specific overrides under the properties of individual groups. If you are only interested in recent posts there is no point in loading older posts, and keeping the number down makes Newsbin far less memory hungry and far more responsive. Note that the difference between "Show Posts" and "Show All Posts" on the groups right-click menu is that Show Posts respects the Display Age value, while Show All Posts ignores it.
• Avoid Multiple Post lists: If you have the Multi-tab Mode option set under Options->Switches then Newsbin allows each group to be loaded into its own Post list. While this can be very convenient when handling small or medium sized groups it should be avoided when handling the very largest groups.
• Avoid multi-group Post lists: Newsbin allows you to load multiple groups into a single Post list. this can be done by organizing the groups into a "Group of Groups" (GoG) and selecting the GoG when opening a Post list. Avoid using this technique on the largest groups.
• Use the Search facility: The Search facility can be used to search groups that are too large to handle in the RAM that you have on your PC. The Search facility reads through the spool files on disk, and only loads those headers that meet your search criteria. The penalty is that this is slower than using the Filter Bar on a Post list, but it does allow very large groups to be searched on machines that do not have enough RAM to load the group into a Post list. There is also an option on the Search tab to limit the number of days worth of headers to be examined - using this will speed up searches just as using the Display Age value does when loading Post lists.

Where are my Downloaded Files?

Once files have been downloaded, there is a Download Path column in the Files tab that has the path where the files are saved. You can also right-click on a file in this list and select "Open Folder" to get to the folder where the files are.

NewsBin has several settings to allow you to specify where your files should be stored. There are various levels and combinations of the following settings that can be used. If you are not sure how they interact, you can always see what destination has been used for a specific file in the Download list by looking in the Download path column.

Note that if you change any of these settings they only apply to new files you add to the download list - they are not applied retrospectively to those files that are already queued.

• Global Download Path
  The main setting is under Options/Setup using the "Download Folder" setting. By default, all files will be stored in this folder, or in a sub-folder relative to this path.

• Group Download Path
  You can configure a different Download Path for each newsgroup or Group of Groups (GoG) by setting the Download Path under the Group Properties. Get there by right-clicking on the newsgroup in the Groups tab and select "Properties". If set, the GoG value over-rides the global value, and the individual newsgroup level over-rides both the GoG and global levels.

• Manual Download Path
  You can also manually specify a download folder when you add a file to the Download list from the Post list using the "Download to Folder" option from the right-click menu. Note that if you use the Manual mode then this will over-ride all the automated ways of setting the folder.

The other point that is commonly required is to get files stored in sub-folders relative to the Download path. There are several settings that can affect this:

• Using Headers or Internet Search:
  In this case the settings under Options->Filename Options are relevant:
  • MP3 Folder Mode
    This setting is specific to MP3 files and means that you want the files to be put into a sub-folder whose name is derived from the ID3V2 tags embedded in the file. This does rely, however, on there being good values set for these tags, and very often this is not the case. It also has the disadvantage that any associated files (e.g. covers or playlists) that are not mp3 files will not end up in this folder.
  • Subject Filename Mode
    This allows the filename to be derived from the subject.
  • Folder Mode
    This setting allows you to specify that the files should be put into a folder named after the group relating to the item selected.

• Using NZB Files:
  In this case the settings under Options->NZB Files are relevant:
  • NZB Folder Modes
    These settings are specific to NZB files. You can specify that a combination of options to control the folder name generated. To help you see what would be generated an example is shown that gets dynamically updated as you change the options. You can also specify whether the prefix should be stripped off the name used to create the folder.

Marking Headers as Old/Deleted

When headers are downloaded by Newsbin they are flagged with a New status when they are written to disk. It is quite normal to want to get this status cleared when you have looked at the headers and decided what to do with them as this then allows you to exploit the Hide Old checkbox on the Filter Bar at the top of Post lists so that you only see posts that are new (or you have not yet dealt with).

the New status of a Post can be cleared in several ways:

• Adding the file to the Download queue
• Marking the file as Old. This can be done by either using the DEL key, or one of the options from the right-click menu. If you have selected a range of posts then the action will apply to all the selected posts. The update of the headers on disk runs as background task so it may take a short while to complete so that if you mark posts as Read and then immediately try to reload the group it may not have completed the task so some of them still show as new.
• Get Newsbin to automatically clear the new status from the disk copy of the headers once it is loaded into a Post list. To do this you need to set the Auto-Mark Old setting under Options->Switches. Newsbin actually clears the new status from the disk headers as soon as it has loaded them into the Post list rather than at the point the Post list is closed.

Note that marking a post as Old does not stop it being loaded into memory - it is still available for viewing if you untick the Hide Old checkbox on the filter bar at the top of the Post List. It also means it is still using memory when the Post list is loaded which could be a factor in very large groups. Post can manually be set back to a New status if you accidentally mark the wrong ones as Old via the option on the right-click menu.

The other facility is to mark a post as Deleted using either SHIFT-DEL or an option from the right-click menu. You would do this if you were absolutely certain that you would never want to see that post again. This does not physically remove it from the local hard disk, but it does stop the Post from ever being loaded into memory again and consuming resources. It will be physically removed from the hard disk if any of the following occur:

• You do a header download and the server reports that the headers concerned are no longer on the server.
• You do a manual Purge to Display Age and the post is older than the Display Age value.
• The post reaches the Storage Age limit (default is 200 days).

Text Files

NewsBin was written to download binary files rather than text files. A basic facility has been implemented to read text files - you are likely to want this to read items like .NFO information files associated with binary files.

Text posts can normally be identified by the fact that they tend to be very small posts (<3K). If you want to read these from within Newsbin you use the Read Post Body feature. This is used as follows:

• Select the post and hit ctrl-r or right-click and select Read Post Body.
• The post will be loaded into the text file viewer that you have set as the default viewer for .txt files at the Windows level. Note this did not happen in the early releases of Newsbin 5 prior to the 5.05 release where you always had to go to the Read Post Body tab as described in the next step.
• The post will also have been added to the Read Posts tab (View->Read Posts to activate if not already visible). You can go there at any time if you want to re-read any recent text posts on which you have used the "Read Post Body" feature. You may have to hit the Reload button on the tab the first time you bring it up.

Binary Files

This is NewsBin's strength, downloading binary files. There are single part and multi-part files, and different encoding schemes. NewsBin hides the complexity of usenet by automatically downloading and decoding individual posts that make up a complete file.

To select a file for download and send it to the Download list you can do any of the following:

• Click the box to the left of the subject.
• Double-Click the post entry
• Highlight the post, right click and select "Add to Download List"
• Highlight the post, and then use the CTRL-E keyboard shortcut

You can also highlight multiple posts using standard windows multi-select methods such as using SHIFT or CTRL keys when highlighting the posts, and then add them all in one go to the Download list using the above methods.

Single Part Posts

Single part files are smaller files that can be downloaded in a single chunk by a single connection. The status column will not appear to be segmented, and the size column is usually measured in KB because of article size limits on usenet posts.

Multi-part Posts

For multi-part posts, the Progress Bar in the Download List is segmented to represent each part that needs to be downloaded. As each part (or chunk) is downloaded, the color of the server being used to download it is displayed in the lower part and green in the upper part. If the download of that chunk completes successfully, the segment will all show as the color of the server that succeeded, while if it fails the segment is turned red.

The top level progress bar in the status column shows the overall level of download completion of the compacted set of files.

You should avoid using any of the system colors for your server (Green, Red, Gray, and Black)

Often when downloading multi-part files you will see that Newsbin appears to start working on a number of files in parallel, and then appear to go back and continue working on the first file. This is expected behavior. What is happening is that Newsbin is getting the first part of each file for duplicate detection purposes. Only when this first part has successfully passed the duplication check does Newsbin start working on the rest of the file. This maximises the usage of your bandwidth as Newsbin does not waste any time downloading any additional parts of a file that is going to fail the duplicate test.

Incomplete Posts

You can create files from incomplete posts by adding them to the download list from the post list. NewsBin will try to download as much as it can. It will stay in the download list unless you have "Assemble Incompletes" selected under Options/Advanced. This only works if there are no errors downloading the parts that NewsBin thinks exists. If there are errors, you can manually assemble the incomplete by right-clicking in the download list and select "Assemble Incompletes".

Assembled incomplete files can be completed by PAR2 files. If you have the Newsbin AutoPAR facility active you will see Newsbin automatically moving some PAR2 repair files from the Wish List tab (assuming there are some available) to the Download tab and downloading them as soon as it encounters Incomplete files. This is done in preparation for repairing the file set it when the remainder of set has been downloaded.

RAR Sets

RARs are the archives and PARs are there to help verify and repair after downloaded. These Rar and Par files will be compacted together to appear as one large file by default until you choose to uncompact or expand the set of files to to add or see the list of individual files.

Split Files

Split files created with Master Splitter or HJ Split will have a numbered extension and will be joined back together in Newsbin if they were posted with a par2 file that was created using the original file.