In the first three entries to this series, I have looked at the IronPython language; introduced the project which will help me learn Python and .Net frameworks; and wrote a multithreaded “Hello World” app. Seen in hindsight, these accomplishments seem small … but, for the first time in the last few months, I feel as though I have a foundation. Things which were once unfathomably difficult have started to become possible. I have even come to appreciate the structure of the MSDN documentation, which is an utterly horrifying development! Given my somewhat slow progress, I am very happy to say that this little tutorial will show how to do something useful: download files!

In this and the next article in this series, I will continue to work on my multithreaded download manager for PodCatcher (my WPF based podcast aggregator). As a way of quick review, there are several things that I would like in this download manager. First, it should allow for multiple downloads at once, with each download on a separate thread. Second, the user should be able to pause and resume downloads. The interface should be straightforward and elegant, yet provide the user with a great number of details including download progress, file name, etc. After reviewing a number of programs, I decided to use the Firefox download manager as a model for the GUI and functionality. I intend to use the Background Intelligent Transfer Service (BITS), available in Windows Vista and XP, as the download engine. This article will focus on the “guts”, namely: Interfacing with BITS through an OpenSource C# Wrapper, called SharpBITS.

Extending IronPython

Before making the jump into SharpBITS and downloads, it’s probably good to review some basic principles. First and foremost: IronPython and its deep integration with .Net. In Part 1 of this series, I dithered on a great deal about the integration between .Net and IronPython. Since then, I’ve learned a little more regarding the way in which this integration was obtained.

IronPython is written in C# as a full implementation of the Python language. As it is implemented in C# and built on the Microsoft Common Language Runtime (CLR), the IronPython engine plugs into the existing .Net/Mono platforms seamlessly. .Net assemblies and classes can be used as native Python classes. Though code is still dependent on the IronPython .dll files, the CLR sees IronPython as a true .Net assembly. This doesn’t just apply to the standard .Net classes shipped by Microsoft as part of the .Net framework. Any assembly written in C# and meant for use in the .Net framework can be easily imported and used. This is hugely convenient as a number of C# wrappers have been written for many important tools, including wrappers for the Insight Toolkit (ITK), which happens to be of particular interest to me as a researcher and scientist.

The .Net assemblies can be imported through the CLR module, the magical component which gives IronPython the ability to talk with other members of .Net. The CLR module is imported with the code:

import clr

After the module has been imported, it is then possible to reference specific assembly files by using the clr.AddReference(‘AssemblyName’) or the clr.AddReferenceToFile(‘AssemblyName.dll’) commands.

BITS and SharpBITS

Now onto the more useful tidbits of this post. As I described in Part 2 of this series, I need a robust download engine. Thus, the use of BITS. It is available on all Windows installations without the need for a user to download additional dependencies. It enables a number of very robust features, which extend far beyond my needs. In short, instead of ask, “Why BITS?” it is much easier to ask, “Why not BITS?”

One of the greatest advantages of BITS is its ability to transfer large files asynchronously between a client and a server. The transfers can occur in the background or foreground. Further, BITS uses idle network bandwidth to transfer the files. The transfer, therefore, has zero effect on the overall user experience.

As implied, it also just so happens that there is a managed wrapper for BITS, called SharpBITS. More importantly, due to the strong relationship between IronPython and the CLR, SharpBITS can be used with zero modifications to the underlying code or without additional wrappers. As a C# wrapper around the BITS service (BITS is implemented in C++), SharpBITS is primarily a communication layer and does not add any important functionality of its own. It is important to note, however, that any module written in a .Net compatible language can be used right out of the box. To quote a much overused mantra: “It just works.”

This is a slightly different experience from using a C Python module. C Python extensions are code written in C against the Python API. While Python “bindings” for a number of important toolkits are available (including Qt, GTK, and ITK; among others), these bindings function in the same way that SharpBITS wraps the BITS service. That is, they are a communication layer. While this is convenient, it also adds complexity to the overall program. The wrappers are additional layers of code that must be maintained outside. Custom code written in pure C# or Visual Basic, in contrast, does not need any “communication layer.” IronPython’s easy extensibility and close relationship to other .Net languages is one advantage over the C Python flavor.

Even so, communication layers can be quite useful. Using the SharpBITS wrapper, it is possible to interact with BITS from any application by adding a reference to the ‘SharpBITS.Base.dll’. In an IronPython application, this is done with the following:

import clr
clr.AddReferenceToFile(‘SharpBITS.Base.dll’)

import SharpBits.Base as SharpBits

The BITS Classes

Instructions to the BITS service are passed through an interface called the BitsManager (a detailed introduction on how to use the SharpBITS wrapper is available here), which only needs to be created once. Downloads and uploads proceed via a BitsJob. A BitsJob may be either an upload, or a download. If it is a download, a BitsJob may contain multiple files. However, an upload may contain only a single file. Nearly all interaction occurs directly with the BitsJob or the BitsManager. This includes enumerating current jobs, creating and canceling jobs, as well as querying a job for its progress.

When using BITS, there are some other important things to keep in mind. Here are a few nuggets I found particularly useful:

1. A BITS job queue is specific to a single user or to the system. The active queue is changed by modifying the property DownloadManager.EnumJobs. In the example below, I will be creating a job for a single user and the EnumJobs property is set to SharpBits.JobOwner.CurrentUser.
2. Jobs are persistent. A single job will remain in the queue until you call BitsJob.Complete() or BitsJob.Cancel(). This initially caused me a great deal of confusion as a job would be complete, yet I would be unable to find the files.
3. A download job runs through the BITS service. It therefore continues even after the program has been exited. Even so, it still needs to be cleared or completed before the files are available for use.

Using SharpBITS – The Interactive Interpreter

To get started with SharpBITS, a BitsManager needs to be created. After the BitsManager has been created, you can then create new or interact with existing jobs. A simple example from the command line with the ipy interpreter might look like this:

downloadJob.Resume()    #7

The code is fairly straightforward. First, a reference is added to the SharpBITS.Base.dll assembly (#1) and the modules are imported into the session (#2). Next, an instance of the BitsManager is created (#3) and the JobOwner is set to the current user. Then, a new BitsJob is created, called downloadJob (#4), and files are added to the queue (#5). The file can be downloaded from http, https, or over a network by using the http://, https:// or file:// tags respectively. The Destination Url (DestUrl) must be a valid file path on the existing computer. Files can only be downloaded to already existing folders. As a reminder, since this job is a download, it can contain any number of files. Upload jobs are limited to a single file.

After the files are added, the JobPriority is set. In this example, the files will be downloaded in the foreground. For files downloaded in the background, BITS will automatically throttle the download speed so that they do not interfere with other programs using the network connection. For downloads in the foreground, no such throttling occurs (#6).

Last, the job is started (#7). When first created, all download jobs begin life in suspension. Once they are started, the job can then be paused by using downloadJob.Suspend() or canceled by using downloadJob.Cancel(). As noted above, the files are released by using downloadJob.Complete().

Conclusion

BITS is a fantastic download engine which can be manipulated by using the SharpBITS C# wrapper. In the next article, I will look at how to start a download job on a separate thread as well as how to bring the various bits and pieces together into a semi-elegant whole. After all, while a command line downloader is useful, we want both the download to start and to see its progress.

_____________________________________________________

For additional details about the integration of C# and .Net, see Chapter 1 of IronPython in Action by Michael Foord, which is available for preview.

Similar Posts:

• Learning IronPython – Part 5 – A Rudimentary Download Manager
• Learning IronPython – Part 3 – A Beautiful Start
• Learning IronPython – Part 6 – From Rudimentary to Functional
• Learning IronPython – Part 2 – A Simple Project
• IronPython – Windows Presentation Foundation Tutorials

Categories: IronPython, Programming
Comments Off