Developer.com content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Preamble
The previous installment[1] introduced the concept of Generic Connection Framework (GCF) and presented the development process of resource adapter to be used in GCF. This time we will discuss the basics of file I/O and network resources handling on BREW from GCF perspective. Two simple modules will be developed, allowing the manipulation of files and HTTP data via the GCF. Implementation details, like multiple readers or support for large data sets manipulation, are further detailed.
GETting …
In its initial version BREW had no ready-made access to Web resources. Starting with version 1.1 a comprehensive IWeb family of interfaces was provided. Sometimes, especially for trivial tasks, IWeb proves to be a little bit too complex and a lighter implementation might save the day. HTTPpipe is just such a small tool. It is a small HTTP wrapper based on TCP sockets and a good reason to discuss asynchronous networking in BREW.
The basic structure of HTTPpipe is:
initConnection(){ INETMGR_OpenSocket ISOCKET_Connect Relinquish_contol_to_AEEShell(write)}write(){ check_connection_status ISOCKET_Write Case AEE_NET_WOULDBLOCK: Relinquish_contol_to_AEEShell(write) Case AEE_NET_ERROR: onError (clean_resources) Else write_data read()}read(){ ISOCKET_Read Case AEE_NET_WOULDBLOCK: Relinquish_contol_to_AEEShell(read) Case AEE_NET_ERROR: onError (cleanResources) Case Reading: read_header. read_data. Relinquish_contol_to_AEEShell(read) Case Done: Clean_Resources}
The initialization phase is simple. A socket is opened and a connection attempt follows. AEEShell will inform the application later, in write(), if the attempt was successful or not. Please note that ISOCKET_Connect forces an asynchronous operation. Once connected we can send the request via ISOCKET_Write. A pattern in asynchronous network operations is AEE_NET_WOULDBLOCK — specifying that the requested activity couldn’t be accomplished but a later try might succeed. At that moment the application can reregister the activity and pass the control again to AEEShell. Once the write succeeds, we are ready for read().
The reading sequence is almost the same as in writing, with one important exception: if data is still available for reading we reregister the reader (the same as for AEE_NET_WOULDBLOCK). This is of course a consequence of the fact that data source is remote and accessing it requires a new roundtrip. Another thing to be mentioned is that the HTTP header has to be read and parsed before reading the actual data.
Asynchronous networking programming is definitely not new and almost all the platforms exhibit an implementation. But from a BREW perspective this is the only model available and, as a bonus, it is an automatic method of keeping the watchdog happy. The system does the entire sequencing job. The only thing the developer has to care about is obeying the above presented pattern.
Implementation Details
ISOCKET_Connect takes a callback of type PFNCONNECTCB, defined as:
typedef void (* PFNCONNECTCB)(void * pUser, int nError);
Our pkCBK structure had to be extended to accommodate this type:
template <class Y, class F>struct pkCBK{ typedef pkCBK<Y,F> pk; void setCallback(Y* y, F fx) { y_ = y; f_ = fx; } static void callbackHandler(void* v) { pk* z = static_cast<pk*>(v); (*z->y_.*z->f_)(); } template <class Z> static void callbackHandler(void* v, Z me) { typedef pkCBK<Y,F> pk; pk* z = static_cast<pk*>(v); F fx = z->f_; Y* y = z->y_; (*y.*fx)(me); } Y* y_; F f_;};
pkCBK is used as in the following sequence:
typedef pkCBK<HTTPpipe, void (HTTPpipe<W>::*)(int)> PI; PI pi_;............pi_.setCallback(this, HTTPpipe<W>::connect);ISOCKET_Connect (,,, (PFNCONNECTCB)PI::callbackHandler<int>, &pi_)) ...
The Data Structrue used in this case (HDS) has some additional parameters — like port and size of the buffer used internally for read/write operations; HTTPpipe itself is a variant of GenericResource, being templatized.:
template <class W>class HTTPpipe;
This design is based on the current ResourceRegistry implementation — elements in Registry are type-distinct instead of type-and-instance distinctiveness. Of course a more capable Registry can be easily implemented using a vector<vector> repository — actually a vector of types, each type keeping a vector of instances. An alternative is to use the Registry as it is but to create additional types if two or more resources have to be used in the same time. This might be interesting for some networking applications — parallel reading from two sources for example. Doing this in GCF is very straightforward — just define a new HDS type, for example HDS1, having a distinct ID. It is then possible to make calls like:
initIO<HTTPpipe<HDS1>, HDS1>(onReadCbk<HDS1>, "192.168.10.100", String("/discrepancies.htm"));initIO<HTTPpipe<HDS>, HDS>(onReadCbk, "192.168.10.101", String("/a.htm"));
The 2 calls will now be processed in parallel and the results accessible in onReadCbk<TYPE>.
Another interesting part of HTTPpipe is readHeader(). It extracts the HTTP response header and makes available the status and the data type. If status is not 2XX success an error is raised.
GET is the only protocol implemented in this example.
… and Filing
A file system is a luxury offered by BREW since its inception. Usually mobile systems offer Database/Records contraptions as storage facilities and not files. Anyway, using the File family of interfaces is quite straightforward and there are various examples illustrating the generic aspects.
GCF has a FileStream resource adapter to deal with files. As file and network resources access might be easily abstracted in terms of streams for example we expect some similarity between FileStream and HTTPpipe. The answer is yes and no.
initConnection(){ IFILEMGR_OpenFile Relinquish_contol_to_AEEShell(read/write)}write(){ check_connection_status IFILE_Write Case Writing: Relinquish_contol_to_AEEShell(write) Case Done: clean_resources}read(){ check_connection_status IFILE_Read Case Reading: read_data. Relinquish_contol_to_AEEShell(read) Case Done: Clean_Resources}
What’s definitely missing from the picture are forcible BREW constructs like ISOCKET_Connect(,,,PFNCONNECTCB) or the asynchronous AEE_WOULDBLOCK pattern. Giving back control to the shell is not requested or enforced by BREW in any way — but is safe. There are 2 potential problems here; large files and slow flash memory. Both of them might create havoc in naove applications, relying on blocking file access. What has to be noted is that IFILE_Read is non-blocking, and there is a IFILE_Readable that can be used to register for later read. Unfortunately there is no write counterpart, and write is blocking so extra care is needed in this case.
Another significant difference is that write and read are omnipotent — a write is a transaction by itself. This makes obviously makes some difference in the implementation.FileStream offers support for operations on large data sets. Actually, the same mechanism can be easily reused by HTTPpipe. In read() operations every data chunk (of size bufSize) can be processed immediately, with no delay, making consumer-producer scenarios easy to implement.
Implementation Details
the Data Structure (FDS) has no particular differences other than support for large data sets operations — a psCBK and a bool isInterrupted member.
psCBK is a command structure working with static functions. It can be used in two ways; as a simple callback wrapper or as a BREW asynchronous callback adapter.
struct psCBK{ typedef void (*F)(void*); psCBK() : y_(0), f_(0), shell_(0) {} bool isActive() { return y_!=0; } void setCallback(void* y, F fx, IShell* shell = 0) { y_ = y; f_ = fx; shell_ = shell; } void operator()() { if (!shell_) { f_(y_); return; } cb_.pfnNotify = (PFNNOTIFY)f_; cb_.pNotifyData = y_; cb_.pfnCancel = NULL; ISHELL_Resume(shell_,&cb_); } private: void* y_; F f_; IShell* shell_; AEECallback cb_;};
Usage :
----.. psCBK p; //passing IShell BREW async cbk p.setCallback(this,onReadChunk,m_pIShell); //ordinary cbk example //p.setCallback(this,onReadChunk); //passing data to FNS ds->fnc = p; ----- //using psCBK if (ds_->fnc.isActive()) ds_->fnc(); //isInterrupted set in onReadChunk if (!ds_->isInterrupted) IFILE_Readable(f_, (PFNNOTIFY)P::callbackHandler, &p_);
Final Notes
Calls involving different resource types can be freely mixed. For example:
initIO<FileStream, FDS>(onWriteCbk,"bubumic", String("123"));initIO<HTTPpipe<HDS1>, HDS1>(onReadCbk<HDS1>, "192.168.10.100", String("/discrepancies.htm"));
and in onWriteCbk:
---..if (ds->error == SUCCESS){ initIO<FileStream, FDS>(onReadCbk<FDS>, ds->address, String("")); initIO<HTTPpipe<HDS>, HDS>(onReadCbk<HDS>, "192.168.10.100", String("/a.htm"));}
This produces an “avalanche” of I/O and networking activity — namely a parallel HTTP transaction and a file write/read. ConnectionPolicy can be embedded in the data structures as a type based policy. The advantage is finer granularity (for example it might make sense to reuse network connections but not to reuse file connections).
Source Code
There are 3 versions of the code due to various compiler portability issues. The 3 packages are not in sync — VC7.1 is the final version and contains all the features presented in this article. As there are no important structural changes migrating to another compiler should be almost automatic.
- Download for Visual C++ 6 – 271 kb
- Download for Visual C++ 7.1 – 310 kb
- Download for ADS1.0.1 – 283 kb
End Notes
[1] A Generic Connection Framework for BREW-
http://www.developer.com/ws/brew/article.php/2242101
About the Author
Radu Braniste is Director of Technology at Epicad. He can be contacted at rbraniste@epicad.com
# # #
