| pg 2 of 2 | Back |
revford blog
Red Elf is about to die!
Useful while it lasted, iPlayer downloads back to Windows only
Thu, 13 Mar 2008 11:32:30 GMT
Well, the BBC have changed something in their setup that stopping my iplayer-download tool working, this means that it's back to streaming only for me, for now.
I'll have a look at getting around this, but I don't see why I should have to.
I'm a British TV Licence holder, I've paid for this the same way as Windows users have, why should I get a lesser service?
And why should I have to fight the BBC website to get it?
They promised this service to us free software users within 2 years, I wrote a script to do it in twenty minutes and they've changed their setup to block it working.
Bad BBC.
iPlayer downloads
Mon, 10 Mar 2008 11:59:47 GMT
I need to point out a couple of things about the iPlayer download script.
First up, not all shows available for streaming are also available for download, it seems mostly random but with a better chance of more recent shows being up there.
If it's not available to the iPhone beta, it's not available to download.
Also these files do not contain DRM, but the BBC wants us to delete files after a month.
This is a way for free software and Mac people to get the same functionality as Windows people, not so we can loot the BBC.
So good people, I'm going to have to trust you to delete the files when you're done.
Please.
iplayer-download
Tue, 03 Feb 2009 17:32:36 GMT
Old and broken code removed.
iPlayer script last tweak for now
Mon, 10 Mar 2008 00:04:09 GMT
Last tiny tweak for now, sorted out to it outputs to a sensible filename.
iplayer-download
Tue, 03 Feb 2009 17:33:03 GMT
Old and broken code removed.
iPlayer download script, cleaned up
Sun, 09 Mar 2008 23:52:04 GMT
Sorted out and cleaned up version of the script. Still calls wget for the work.
Pretending to be an iPhone isn't needed, so I can use LWP::Simple and not have to worry about the more complex LWP::UserAgent.
This trims the fat nicely and makes the whole thing read better.
I'll look at modules for downloading the file within the Perl script, rather than by calling wget soon.
iplayer-download
Tue, 03 Feb 2009 17:31:51 GMT
Old and broken code removed.
iPlayer download script
Sun, 09 Mar 2008 21:06:34 GMT
Okay, this is still a quick hack, but it works. Perl this time, should run on anything sensible.
It claims to be an iPhone, reads the show page, grabs the iPlayer ID, then calls wget to download it.
I'll work on this more later, but it's a working tool for now.
iplayer-download
Tue, 03 Feb 2009 17:31:01 GMT
Old and broken code removed.
iPlayer IDs
Sun, 09 Mar 2008 19:37:47 GMT
Seems I'm not exactly right on the iPlayer IDs. These don't always match.
I'm working on it...
iPlayer using only Free Software
Sun, 09 Mar 2008 19:02:52 GMT
BBC iPlayer on GNU/Linux without Flash using only free software
Thank you.
I junked up a quick bash script to download shows from the command line. Seems to work only on the more recent shows as it's taking shows from the iPhone version of the iPlayer.
iplayer-swipe
To use it, find the iPlayer ID of the show, so for example the URL of the current episode of Love Soup is:
http://bbc.co.uk/iplayer/page/item/b009gf52.shtml
So the ID is b009gf52, just enter:
iplayer-swipe b009gf52
And wget should download the show.
Update
Tue, 03 Feb 2009 17:20:59 GMT
The BBC updated the iPlayer and stopped this script working a long time ago, so it's gone.
sitebuild-log-check updated
Sun, 09 Mar 2008 15:06:02 GMT
Updated the log checker, sitebuild-log-check to give me a breakdown of the OS being used, as well as the browser.
Currently it counts Windows XP, Windows Vista, Mac OS X and Linux/Unix users.
Stats so far are very heavily weighted to Windows XP, Mac OS next, then Vista and Linux/X11 users are as rare as hen's teeth.
The rest of sitebuild-update, updated
Sun, 02 Mar 2008 17:50:15 GMT
The rest of the sitebuild-update scripts now updated. Stripping file extensions from post filenames used to create the links from the RSS to the HTML.
sitebuild-update-rss updated
Sun, 02 Mar 2008 16:58:43 GMT
Update to sitebuild-update-rss, a typo was causing the links to be incorrectly added to the RSS feed.
All sorted now.
Sitebuild scripts updated
Thu, 28 Feb 2008 18:32:03 GMT
I've been using an updated version of the site building scripts recently, all seems to work fine so here is the new version.
There are two shell scripts that I run, either sitebuild-update for just the blog, index and RSS feed. Or sitebuild-total for the whole site.
Either script then uses ncftpput to upload the files.
These call the actual working scripts, written in Perl.
The three files sitebuild-update-* each build a different page using the blog posts. sitebuild-update-html is complete blog archive page, update-log. sitebuild-update-index is the main page here. sitebuild-update-rss is the RSS news feed.
sitebuild-contfiles creates all the other pages here, by adding the headers, scripts, CSS and metadata to a set of template files.
In addition, I've added a log file checker. Again a combination of Shell and Perl scripts.
sitebuild-log calls two scripts, first a shell script, sitebuild-log-grab, then a perl one, sitebuild-log-check.
The shell script downloads the log files using ncftpget, then uses gzip to extract them. After that, it's over to the perl script.
sitebuild-log-check tries to get some helpful or at least interesting info about the usage of the site, it uses a perl module to do whois lookups so I can see where in the world the visitors are, then tells me about browser usage, which search engines and other robots are visiting and if I'm getting any 404 errors. The whois isn't perfect, but gets most addresses.
It did show me a mountain of 404s, which I quickly found were looking for favicon.ico and robots.txt, so adding them solved most errors. The others were links to older pages or pages and images that have been removed.
The script also removes information about images linked from the forums I post on, as for stats I'm only interested in people who actually visit this site, not look at my posted images on another.
Most worrying error seems to be that there are people out using Windows Vista (aka NT 6.0), Also there are some people out there using Windows 98 still.
Along with that, there are lots of people using IE6. When there is no good reason to.
Advice:
- If you're using IE6, upgrade to Firefox. It's free and much, much safer.
- If you're using Windows 98 or Vista, upgrade to Linux. It's free and much, much safer.
Get Firefox
Get Ubuntu LinuxHere is an example day's output:
Results for Tue, 26 Feb 2008 22 unique human IPs with 221 dupes. 208.111.154.15 - - [] 75.170.177.133 - - [] 63.167.255.155 - - [] 195.93.60.67 - US - AOL Inc [AOL International Operations, Europe] 195.93.60.68 - US - AOL Inc [AOL International Operations, Europe] 195.93.60.69 - US - AOL Inc [AOL International Operations, Europe] 195.93.60.74 - US - AOL Inc [AOL International Operations, Europe] 195.93.60.70 - US - AOL Inc [AOL International Operations, Europe] 193.130.15.240 - GB - PIPEX-BLOCK1 [UK PA route] 200.238.68.114 - - [%The use of the data below is only permitted as ibed in] 78.150.43.198 - GB - Opal Telecommunications Internet Service Provider [Opal-Net Autonomous System] 195.140.123.22 - DE - SI-NORD-NET1 [Sparkassen Informatik GmbH Muenster] 90.45.138.11 - FR - [BSBOR257 Bordeaux Bloc 2] 198.241.217.15 - - [] 209.226.225.16 - - [] 79.145.172.19 - ES - Provider Local Registry [RIMA (Red IP Multi Acceso)] 86.150.70.200 - GB - IP pools [BT Public Internet Service] 66.249.84.69 - - [] 81.108.176.98 - GB - NTL-UK-IP-BLOCK [NTL-UK-IP-BLOCK] 66.249.84.67 - - [] 86.162.210.216 - GB - IP pools [BT Public Internet Service] 71.196.40.60 - - [] 11 different browsers. Firefox hits 55 Explorer hits 137 Safari hits 0 Opera hits 0 Mozilla/5.0 X11; U; Linux i686 x86_64; en-US; rv:1.8.1.11 Gecko/20080109 Charlotte/0.9t; http://www.searchme.com/support/ Mozilla/4.0 compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322 Mozilla/4.0 compatible; MSIE 7.0; AOL 9.0; Windows NT 5.1; .NET CLR 1.1.4322 Mozilla/5.0 Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.12 Gecko/20080201 Firefox/2.0.0.12 Mozilla/4.0 compatible; MSIE 6.0; Windows NT 5.0 Mozilla/5.0 Windows; U; Windows NT 5.1; fr; rv:1.8.1.12 Gecko/20080201 Firefox/2.0.0.12 Mozilla/4.0 compatible; MSIE 7.0; Windows NT 6.0; Mozilla/4.0 compatible; MSIE 6.0; Windows NT 5.1; SV1 ; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; .NET CLR 1.1.4322 Mozilla/5.0 Windows; U; Windows NT 5.1; ca; rv:1.8.1.12 Gecko/20080201 Firefox/2.0.0.12 Mozilla/4.0 compatible; MSIE 7.0; Windows NT 5.1; YPC 3.2.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; yplus 5.3.04d Mozilla/4.0 compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322 Mozilla/4.0 compatible; MSIE 6.0; Windows NT 5.1; SV1; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322 4 different robots. msnbot/1.1 http://search.msn.com/msnbot.htm Mozilla/5.0 compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp Mozilla/5.0 compatible; Ask Jeeves/Teoma; http://about.ask.com/en/docs/about/webmasters.shtml Mozilla/5.0 compatible; Googlebot/2.1; http://www.google.com/bot.html 1 404 errors.
Less JS, more offline scripted building
Mon, 11 Feb 2008 14:34:02 GMT
More behind the scenes updates to how the site works. The page title, which CSS is needed, metadata and scripts are now added to the pages with a script run on my machine, leaving less for deco.js to do.
This maintains, even improves, site coding simplicity for me and requires less work for the client end. This means pages should load faster.
The Perl that makes it work is here, sitebuild-update-contfiles. The contfiles it talks about are the content of the pages, in basic HTML, leaving out the head, body, html doctype etc.
| pg 2 of 2 | Back |
HTML