PermaLinkProcessing RSS In LotusScript - Part 1: Retrieving XML Via HTTP
03:14:59 PM

A couple of recent posts, one by Vince, and one by Lance, have reminded me to start publishing pieces of my bloggregator source code. I will eventually release the whole thing as open source. There's a little more functionality that I want to add first, and I also have to learn more about open source licensing before I take that step. In the meantime, I'll start posting articles revealing pieces of the source. This first article is inspired by Vince's question about posting XML via Domino.

When I first read Vince's post I thought he was talking about the reverse getting XML into Domino over an SSL connection. I realize now that his concern is the opposite: sending XML from Domino to a web server over an SSL connection. Although I haven't done that, I the tool that I use to retrieve XML into Domino, the WinHTTP COM object, can also be used to post XML from Domino. Here is the GetXML function that I use in bloggregator.

Function getXML(theURL As String) As String ' Use WinHTTP COM object to retrieve an XML page Dim http As Variant Dim temp As Variant Dim xml As String Dim xmlString As String Dim i As Double Dim statusNum As Double Dim statusText As String Dim errmsg As String ' use error trapping to account for two possible versions of WinHTTP that might be installed On Error Goto WinHTTP51_Error Set http = CreateObject("WinHTTP.WinHTTPRequest.5.1") Goto Continue_WinHTTP WinHTTP51_Error: ' failed creating WinHTTP 5.1 object, so try WinHTTP 5.0 Resume Try_WinHTTP50 Try_WinHTTP50: On Error Goto WinHTTP5_Error Set http = CreateObject("WinHTTP.WinHTTPRequest.5") On Error Goto 0 Continue_WinHTTP: On Error Goto HTTP_Error ' Make an HTTP request to get the XML from the remote server temp = http.Open("GET", theURL, False) Call http.Send() statusNum = http.status statusText = http.statusText xml = http.responseText xmlString = Cstr(xml) On Error Goto 0 Dim c1 As Long Dim c2 As Long Dim c3 As Long ' massage the XML to make sure that DOM parser won't barf on it c1 = Instr(xmlString,"<!DOCTYPE") If c1 > 0 Then c2 = Instr(xmlString,"<!ENTITY") If c2 > 0 Then c3 = Instr(c2,xmlString,">") c2 = Instr(c3+1,xmlString,">") Else c2 = Instr(c1,xmlString,">") End If For i = c1 To c2 Mid(xmlString,i,1) = " " Next End If getXML = xmlString Exit Function HTTP_Error: errmsg = "HTTP Error: " + Cstr(Err()) + " " + Error(Err()) + " URL: " + theURL Print errmsg getXML = "" Resume Bail WinHTTP5_Error: ' had a problem with WinHTTP COM object - assume no access errmsg = "WinHTTP Error: " + Cstr(Err()) + " " + Error(Err()) + " URL: " + theURL Print errmsg getXML = "" Resume bail Bail: Exit Function End Function
This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at

I have used variations of this code in several other projects, including one that used SSL connections, which is why Vince's post brought it to mind. Using SSL with WinHTTP is very straightforward, just a matter of adding an 's' to the URL -- unless you want to do authentication, but that's too painful an experience to blog about. SSL isn't relevant here. At least I don't think it is. I've never run into an SSL-protected RSS feed yet.

Let's look at the code. WinHTTP is a simple interface for doing http communications from COM-enabled programming languages. We're going to do an http GET request for a URL that was passed into the function in the argument theURL. You can do http POSTs and various other operations with it almost as easily. You need some minimal knowledge of the workings of the http protocol, but nothing really that you can't get from the RFC. To use WinHTTP, first we instantiate an object of type WinHTTP.WinHTTPRequest.5.1 or WinHTTP.WinHTTPRequest.5 into a variant called http. The only complication is dealing with the fact that either of the two different versions of WinHTTP may or may not be installed, so we have to trap errors and deal with them gracefully. Next, we call the Open() and Send() methods of the WinHTTP object, and check the status property. We trap errors at this stage, too, just to be sure (and when doing SSL work, this did prove to be necessary in some cases). Finally, we check for the and tags and remove them. Why do we do this? Because far too many RSS feeds are... let's just say... "overly optimistic" in what they promise, and their failure to validate causes the NotesDOMParser object to throw an error. If you're not dealing with RSS, you can take this part of the code out. What you'll have left is a general purpose routine that retrieves XML (or just plain HTML for that matter) from a web server. You may want to do some error logging rather than just using print statements, but other than that it's production-ready as far as I'm concerned. I've been using it for two years, pulling in more than 200 feeds, running every two and a half hours, and it has never caused a problem.

The downside of using WinHTTP is that it is a Windows-only solution. If you're building an RSS processing application, and you want it to run on a Notes client for Mac, or a Domino server for Linux, iSeries, AIX, zSeries or Solaris, you're have to use Java instead. But if Windows is good enough for you, the above code will get you started.

Part 2, coming soon, is inspired by Lance's observation about problems in many XML feeds.

This page has been accessed 3799 times. .
Comments :v

1. Andrew Barker02/11/2005 09:03:16 AM

Rich -
I had excitement right up to the point where you said "unless you want to do authentication, but that's too painful an experience"... so, really, how painful is that part?

Would it still be painful if I can hard-code the username and password?

I have some non-sensitive xml I would like to retrieve and format from another box, but I must authenticate and this facet has put me over my limit for this hobby code.

Any insights would be great. Thanks.

ps - I like the new layout, it's reminding me to change your uri in my linxRoll.

pps - while I am on the topic, would you be willing to change the path of my site in your's? (it is just "home" now)

2. Lance Spellman02/11/2005 05:43:40 PM

Thanks for the insight. I was curious what you were doing to pull in the feeds. We went the Java route which turned out to be real interesting. Initially, we wanted it to be totally native to the classes that Notes provided, either in Notes.jar or (the antiquated) NotesXSL.jar. Eventually we did end up adding some external code for an HTTP client reader b/c of Java 1.3.1's issue with hanging on a timed out URL.

This led to an interesting email from an IBMer last Spring. He'd reported a bug, and as I dug through the code, I determined the only way he could be getting it was from the Factory class pulling in a different Transformer. And that could only happen if...he was using new code. So I pinged him and queried if he was on the Domino 7 team. Sure enough, he was using an early Beta Notes 7 client where they've replaced all the NotesXSL stuff with Apache Commons code (hurrah!).

So, there will need to be a code update for 7, but the Native Solution for 7 should be a lot simpler and efficient than 6.x I'm also looking forward to porting this to Workplace Client.

3. Tom Rau06/28/2006 10:48:39 AM
Homepage: http://NA


Great article. Could you tell me how to do a "POST" using WinHTTP. I looked at the Microsoft object and only saw a "PUT".


4. Pablo Solano05/01/2007 12:18:11 PM

I'm getting this error:
using your code:
Call getxml("")

HTTP Error: 213 %s: %s URL:

Can you tell me why?

This URL is just an example. I need to read an XML from url and compare their data against my Notes data. Any clue?

5. Patrik Plesky05/28/2009 04:42:19 PM


In my case,
HTTP Error: 213 %s %s was caused by setting authentication before calling Open()

Blocked Response!11/06/2009 09:40:57 AM

This response from IP Address was blocked by the owner of this blog.

7. vesoftware11/05/2013 10:18:03 PM

Agen Bola Promo 100% SBOBET IBCBET Casino Poker Tangkas Online
ITUPOKER.COM AGEN POKER ONLINE INDONESIA TERPERCAYA : Toko belanja online murah, Promo heboh jual barang hanya Rp 1,-

8. 51316914103/03/2016 08:57:24 PM

9. ayodon03/27/2016 09:28:24 AM

Pernah denger danau ranu kumbolo? Ya, danau indah ini terletak di gunung semeru dan masuk dalam kawasan taman nasional bromo tengger semeru. Danau ranu kumbolo merupakan daya tarik para pendaki untuk mengunjungi gunung semeru. Jika anda membutuhkan paket wisata bromo dan danau ranu kumbolo tanpa harus menuju puncak gunung semeru maka silahkan pesan guide dan porter pada kami. Info lengkap tentang paket wisata murah ke bromo dan danau ranu kumbolo baca di paket wisata bromo ranu kumbolo

Madas Trans merupakan perusahaan jasa sewa menyewa mobil yang terletak di empat kota besar yaitu di Surabaya, Jogja, Malang dan Denpasar Bali. Tidak hanya sekedar sewa mobil saja, Madas Trans juga menyediakan beragam paket wisata murah yang akan siap melayani perjalanan wisata anda di 4 kota besar tersebut khususnya dan umumnya untuk daerah Jawa Timur karena kantor pusat sewa mobil Madas Trans terletak di Kota Surabaya.

Jika ingin memesan jasa sewa mobil dan paket wisata murah tersebut anda bisa menghubungi nomor telkomsel 085204788678 dan tentunya pelayanan yang ramah dari customer service akan menyambut anda dengan senang hati. Banyak armada pilihan sewa mobil di madas trans tersebut seperti sewa mobil avanza di jogja sewa mobil innova, rental mobil xenia, sewa mobil elf pariwisata baik short atau pun long, sewa hiace commuter dan sewa bus mini serta rental bus pariwisata.

Sopir sewa mobil yang disediakan di Madas Trans akan menjadi guide perjalanan wisata Anda. Motto utama dari berdirinya Madas Trans tersebut adalah “YOUR COMPLETE GUIDE” dan kami pastikan Anda akan merasakan kepuasan setelah menggunakan jasa sewa mobil di tempat kami karena kami sudah lama malang melintang dalam usaha sewa menyewa mobil dan sudah berdiri sejak tahun 2013 silam. Jika ada yang kurang paham dan timbul pertanyaan terkait dengan sewa rental mobil dan paket wisata yang kami tawarkan maka silahkan hubungi CS kami. Jangan sungkan dan jangan ragu.

10. yuli04/27/2016 04:45:45 AM

Info sewa mobil madiun :

11. lllllyuan05/16/2016 12:03:12 AM


12. chenyingying10/17/2016 12:19:10 AM

13. sewa mobil11/16/2016 01:43:02 AM

thanks for sharing article

agen pulsa flat
honda kediri
catering solo
sewa mobil madiun
paket wisata jogja
sewa mobil jogja
dhea tour transport jogja
sewa mobil banyuwangi
sewamobil malang

14. dongdong806/28/2017 11:42:48 PM

15. chenlixiang08/11/2017 09:02:15 AM lauren.html

16. ylq jake08/17/2017 04:50:53 AM,,,,,,,,,,,,,,,,,,,,,,,, ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Enter Comments^

Email addresses provided are not made available on this site.

You can use UUB Code in your posts.

[b]bold[/b]  [i]italic[/i]  [u]underline[/u]  [s]strikethrough[/s]

URL's will be automatically converted to Links

:-x :cry: :laugh: :-( :cool: :huh: :-) :angry: :-D ;-) :-p :grin: :rolleyes: :-\ :emb: :lips: :-o
bold italic underline Strikethrough

Remember me    

Monthly Archive
Responses Elsewhere

About The Schwartz


All opinions expressed here are my own, and do not represent positions of my employer.