"Eric" <BorgMotherShip@AliensR_US.org> wrote in message
news:EbadncCu64c3F0HfRVn-(E-Mail Removed)...
> I want to get a web page containing my stock grants.
> The initial page is an https and there is a form on it to
> fill in your username and password and then click "login"
> I played with python's urlopen and basically it complains "your browser
> doesnt support frames" meaning the urlopen call makes it unhappy somehow.
The page you're trying to open employs the use of frames, which your urlopen
doesn't support.
> Is it reasonable to think i can build a script to login to this secure
> website, move to a different page (on that site) and download it to disk?
Yes, except you'll want to go directly to the page that provides the data of
interest, rather than navigating after the authorization. One can determine
this in Firefox by right-clicking the frame of interest, then "Show only
this frame" and taking note of the URL that gets displayed.
> Any ideas as to how to proceed with this quest?
It depends on the format of the page contained in the frame that is
delivering the data.
If one needs to simply download that HTML page, then (e.g.):
#!/bin/bash
tempfile=/tmp/filename
echo 'To:
(E-Mail Removed)in
Subject: Daily stock report
Cc:
(E-Mail Removed)
Content-Type: text/html; boundary="-- boundary --"
' > $tempfile
/usr/bin/wget -O- --http-user=USER --http-passwd=PASS \
https://the/exact/URL >> $tempfile 2> /dev/null
/usr/sbin/sendmail -t < $tempfile
# end script
If the page is a "form" that requires specific input, then you'll have to
examine the page source to get the field (-F) names, and the name of the
other incidentals, then use cURL
http://freshmeat.net/projects/curl/ to
submit the form with the correct field (-F) values, e.g.:
datetoday=`date '+%D'`
/usr/bin/curl -F "Starting Date"=$datetoday \
-F "Ending Date"=$datetoday
https://the/exact/URL > $tempfile 2>
/dev/null
You can handle the output depending on whether it's a csv file, HTML, ascii
text etc.
Similar Threads
Linear Mode
