Castle Paradox Forum Index Castle Paradox

 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
 Gamelist   Review List   Song List   All Journals   Site Stats   Search Gamelist   IRC Chat Room

Mirror-friendly game index

 
Post new topic   Reply to topic    Castle Paradox Forum Index -> Site Suggestions
View previous topic :: View next topic  
Author Message
Bob the Hamster
OHRRPGCE Developer




Joined: 22 Feb 2003
Posts: 2525
Location: Hamster Republic (Southern California Enclave)

PostPosted: Fri Oct 07, 2011 8:17 am    Post subject: Mirror-friendly game index Reply with quote

I haven't been running the gamelist mirror in a long time. Even though it only ran once a month, I know it put a strain on the site (and on mine too, I think)

I would love to be able to update a mirror of the game list incrementally. Over at Slime Salad, Mogri implemented a mirror-friendly dump of the gamelist, and I was wondering if you would be willing to add a similar mirroring tool here to CastleParadox?

The format Mogri used looks like this:

Code:

Game Name
Author Name
http://full/url/to/game
Screenshot or Download Filename
http://screenshot/or/download/url
last_modified_timestamp
Screenshot or Download Filename
http://screenshot/or/download/url
last_modified_timestamp



http://www.slimesalad.com/forum/gamedump.php

I know the gamelist is structured differently here, so the exact same fields might not make sense, but if you could provide something somewhat similar, I could easily write a script to mirror only the files that had changed since the last mirror.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
TMC
On the Verge of Insanity




Joined: 05 Apr 2003
Posts: 3226
Location: Matakana

PostPosted: Fri Oct 07, 2011 10:29 am    Post subject: Reply with quote

I couldn't help myself!

http://tmc.castleparadox.com/gamelist.py

Aside from some very minor playing around with PHP, this is the first time I've ever done anything like this.
_________________
"It is so great it is insanely great."
Back to top
View user's profile Send private message Send e-mail MSN Messenger
Inferior Minion
Metric Ruler



Joined: 03 Jan 2003
Posts: 741
Location: Santa Barbara, CA

PostPosted: Fri Oct 07, 2011 11:19 am    Post subject: Reply with quote

Haha, I was just finishing that up myself. TMC, you forgot to allow for JPEG screenshots Happy

James, does this work for you? I've annotated the list elements using the title attribute for clarity: http://castleparadox.com/gamelist_mirror.php

Cheers,

~IM
_________________
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address MSN Messenger
TMC
On the Verge of Insanity




Joined: 05 Apr 2003
Posts: 3226
Location: Matakana

PostPosted: Fri Oct 07, 2011 12:11 pm    Post subject: Reply with quote

Opps, that regular expression was just a placeholder!

You forgot the most important part: the mtimes!

Was linking to downloadcounter.php accidental?
_________________
"It is so great it is insanely great."
Back to top
View user's profile Send private message Send e-mail MSN Messenger
Inferior Minion
Metric Ruler



Joined: 03 Jan 2003
Posts: 741
Location: Santa Barbara, CA

PostPosted: Fri Oct 07, 2011 1:01 pm    Post subject: Reply with quote

Thanks for pointing out the missing time information. I was stripping out as much information from the game list page as I could. Also, if you look at the download link, you'll notice it has &mirror=true in the URL. This allows the download page to do the filename translation while skipping the actual download count incrementing.

Cheers,

~IM
_________________
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address MSN Messenger
TMC
On the Verge of Insanity




Joined: 05 Apr 2003
Posts: 3226
Location: Matakana

PostPosted: Mon Oct 10, 2011 2:09 pm    Post subject: Reply with quote

I'm confused. downloadcounter.php?game=X&mirror=true doesn't download the game, it just leads to a broken version of the game listing, where the download link isn't tagged by mirror=true. Or is the download count incremented by something like an onclick action which isn't present on that page? Anyway I assume the mirror script would prefer a direct download link: would downloading via that increment the download counter?
_________________
"It is so great it is insanely great."
Back to top
View user's profile Send private message Send e-mail MSN Messenger
Bob the Hamster
OHRRPGCE Developer




Joined: 22 Feb 2003
Posts: 2525
Location: Hamster Republic (Southern California Enclave)

PostPosted: Mon Oct 10, 2011 2:43 pm    Post subject: Reply with quote

I guess I can strip out the html with a regular expression, but multiline regexes are a huge pain. For my own purposes this would be much easier if it was a minimalist plain text format.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Inferior Minion
Metric Ruler



Joined: 03 Jan 2003
Posts: 741
Location: Santa Barbara, CA

PostPosted: Mon Oct 10, 2011 3:43 pm    Post subject: Reply with quote

Thanks TMC. I forgot that I was using a test page when creating the game list dump. My test page would link to a test download counter page (see FFH's download counter for an example: http://castleparadox.com/downloadcounter.php?game=37 - may take a minute to load). I had updated download.php to support a "mirror" url query parameter so that you would get back a proper filename, not #.zip, while avoiding incrementing the download count. I've fixed the issues with the links in the page.

James, I've made the page output plain text wrapped in pre tags by default.

Cheers,

IM
_________________
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address MSN Messenger
TMC
On the Verge of Insanity




Joined: 05 Apr 2003
Posts: 3226
Location: Matakana

PostPosted: Tue Oct 11, 2011 7:13 am    Post subject: Reply with quote

Oh! I had mistaken that table on downloadcounter.php for some kind of misformatting of the download stats histogram. That's pretty interesting. I wonder what could be mined out of those logs... if you wanted to find correlations (ie "also downloaded"), you would want to use the original data with more precise timestamps...
_________________
"It is so great it is insanely great."
Back to top
View user's profile Send private message Send e-mail MSN Messenger
Display posts from previous:   
Post new topic   Reply to topic    Castle Paradox Forum Index -> Site Suggestions All times are GMT - 8 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group