|
Post by RielCZ on Jan 31, 2022 14:10:41 GMT -5
I have a working BC scraper! It's been downloading for about 14 hours alphabetically by pet (saving the top 3 for each week) and is currently on Cybunny. So, it'll be some time yet, but hopefully by the end of the week the BC will be saved. (Barring any missing pet pictures, and there are a few more than I expected heh.) I still want to develop a script to save the Random Contest, but beyond that... are there any other creative contests that should be archived?
|
|
|
Post by June Scarlet on Jan 31, 2022 22:38:58 GMT -5
That's amazing Riel!
|
|
|
Post by Carolyn on Feb 1, 2022 22:04:26 GMT -5
Riel, you're incredible. ;_; Words cannot express how amazing this is and how awesome your efforts are. You're making sure that even without the website (although I hope the day never comes), all of this amazing creative work isn't lost to the ages. That in itself is really heartwarming and wonderful.
Unsure if this was already done, but poetry gallery? Storytelling contests? Maybe some pet spotlights?
Are there any Neopian-run NT sort of things from over the years?
|
|
|
Post by RielCZ on Feb 3, 2022 17:36:51 GMT -5
Riel, you're incredible. ;_; Words cannot express how amazing this is and how awesome your efforts are. You're making sure that even without the website (although I hope the day never comes), all of this amazing creative work isn't lost to the ages. That in itself is really heartwarming and wonderful. Unsure if this was already done, but poetry gallery? Storytelling contests? Maybe some pet spotlights? Are there any Neopian-run NT sort of things from over the years? Aww, thank you so much Carolyn. ^^ Yeah, I hope the day never comes too, but... it's better to be ready than not. I know how important/influential these creative contests are/were to so many people, myself included. Besides the NT, I do have the Poetry Gallery, the STC, the Art Gallery, the Pet Spotlight, and the Petpet Spotlight archived. I do run my scraping tools every few weeks to pick up the most recent entries in the active contests. I have the CC mostly archived as well, but I know there were a few errors with that script and not every contest was saved; I need to look through the logs and redownload where necessary. As an update on the BC scraper, it is currently on Uni, 2003. For anyone curious, for the BC, the average size of all the files/images in each Pet's folder is around 100 MBs.
|
|
|
Post by RielCZ on May 14, 2022 22:40:37 GMT -5
Not really an update per se, but I am still saving the contests as they go by.
...Except for very recently, since the frequency of the "Stackpathing" as of late is impacting the scripts actually getting access to the webpages. (I noticed this too when I tried to generate one of those Review Thread posts earlier today and got a fatal error.)
Hopefully they tone the site verification thing down soon (though I've noticed that it can be better at different times of the day, at least in the past). If not, I'll have to (at some point) implement somewhat more complex measures for scraping.
|
|
|
Post by RielCZ on Aug 3, 2022 1:11:28 GMT -5
Just in time for the MD20 collab, I modified my NT download script to get around the "site verification" issue and I can now download the NT again. I'll try to roll that out for my other contest-save scripts, too. Also, I never reported it, but with some of Mostly Harmless (flufflepuff)'s help a couple months ago I realized how to scrape the creative contests in other languages. I got the Poetry and STC... are there more?
|
|