Skip to main content

Topic: My Opera Backup (Read 9594 times)

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
My Opera Backup
As most of you know, I'm currently running a Python script with the intent of backing up all of the My Opera forums, or at least about as close to it as possible.

I already shared a few hundred thousand posts using My Opera itself, but obviously that would be self-defeating. I considered torrents. They have many advantages, not least of which is that the more popular they are the faster they (ought to) go. But it would be more likely to be a constant drain on my Internet volume, which is currently limited to a minuscule 100GB. (I can make do with it, and I don't want to pay more.) I don't think the ISPs here understand that even buying a game on Steam these days might use up 100GB within a half hour.

Anyway, the best solution seems to be sharing it all on Wuala. Downloading all the files should be easy through the Opera Links panel, or some equivalent extension on another browser. Sharing it from this very server is also an option, but in the unlikely case that there's a lot of interest I figure it's better to have it elsewhere.

Download backup files from Wuala

I won't share too much until I've done a data integrity check and some pruning. These initial files are mostly to (hopefully) get someone interested in processing the available data in some way. To that end, I penned down a few suggestions.



Some technical information: 3,331,270 items, totalling 4.7 GB.

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #25
If you want to do some quoting from My Opera in these next two days, here's a little script that'll help:

You could save this as a .user.js in the appropriate folder:
Code: [Select]
var links = document.querySelectorAll('a[href^="findpost.pl"]');
for (var i = 0; i < links.length; i++) {
  var comment_id = links[i].href.match(/\d+/);
  var topic_id = location.href.match(/id=([0-9]+)/)[1];
  links[i].outerHTML +=' <a href="http://my.opera.com/community/forums/reply.dml?action=quote&commentid='+comment_id+'&id='+topic_id+'">quote</a>';
}


Or use it as a bookmarklet:
Code: [Select]
javascript:(function(){var%20links=document.querySelectorAll('a[href^=%22findpost.pl%22]');for(var%20i=0;i%20%3C%20links.length;i++){var%20comment_id=links[i].href.match(/\d+/);var%20topic_id=location.href.match(/id=([0-9]+)/)[1];links[i].outerHTML+='%20%3Ca%20href=%22http://my.opera.com/community/forums/reply.dml%3Faction=quote%26commentid='+comment_id+'%26id='+topic_id+'%22%3Equote%3C/a%3E';}})();

Re: My Opera Backup
Reply #26

The last file is now online.

Very cool!!!  8)  Thank you SO MUCH for doing this. Frenzie!!!   :D

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #27
No problem. If you come up with something cool to do with the data, please share it here (or on Github.) I wrote down a few quick suggestions here.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #28
Did anyone back up the widgets? I forgot :(
There are three I used to like and use:
- Frenzie's weather widget
- Some other weather widget, a more recent development
- a calendar widget with holidays, reminders, and translations

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #29
- Frenzie's weather widget

That'll be tough to find 'cause I never made any of those! :)

Are you perhaps thinking of the weather Unite app made by lucideer as a proof of concept? BS-Harou also made a weather speed dial extension.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #30
No, not unite app. Widget. Was there none uploaded by you? There was a blue weather widget that I for some reason associated with you.

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #31
I never made any widgets, or at least not publicly. Privately I'm sure I made one or two "hello world" types of widgets. :P 'cause yeah, they're basically just a type of web pages anyway.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #32
Whoever made it, it was a cute widget. Any sanctuary or haven of Opera widgets anywhere?

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #33
johnnysaucepn?

If so, it looks like you can still get it here. Also see here.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #34
Yup, that's the one. Thanks :)

And sorry for confusing the names. I confuse names in real life too. To me names seem totally random and undescriptive, always did. I tend to not use people's names in communication, because it's embarrassing to mix them up...

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #35
And I tried it immediately. I remembered I uninstalled Johnny's widget because it broke at some point. It can be installed, but doesn't work (should be fixable by redirecting to some other source for meteorological data). The other weather widget I liked, it still works http://macdownload.informer.com/multiweather/download/

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #36
Opera IRC is still up, btw. By mistake, I suppose.

And so is help.opera.com. Is it a legitimate idea to copy Opera's help pages to operawiki.info?

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #37
Is it a legitimate idea to copy Opera's help pages to operawiki.info?

Probably not. I'd just run wget on it with an Opera 12 user agent or some such.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #38
The user agent is unnecessary, I think. The help pages were always accessible with any browser.

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #39
In an oddly familiar turn of events, Wuala will stop offering free space from December 31, 2014. Are there any other suggestions for sharing a few GB of data? I suppose I could always turn it into a torrent, but then availability as well as upload speed would be more limited.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #40
If you don't know, then I doubt anyone else knows about such sharing place. What's the max limit of Dropbox? Does it do such sharing anymore?

But torrent always seemed a reasonable alternative sharing option. Publish it in Pirate Bay or such and some fans (such as myself) might remain seeding.

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #41
While I could easily put it on a shared folder in Dropbox (I seem to have 20 GB available these days), I don't want that propagating across all of my computers. Also I believe it's hard to download all files from a Dropbox folder the way you easily can in Wuala.

Anyway, I'm looking into setting up some kind of webseed at the moment to get the best of both worlds. You can give it a try using this torrent.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #42

You can give it a try using this torrent.

So, I gave it a try this weekend. It has been stalling at 0.4%. It's precisely weekends when I can pull and share torrents...

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #43
Weird, what torrent client are you using? The webseed is supposed to kick in when there's a lack of peers.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #44
qBittorrent

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #45
Hm, me too. Let's see, it downloaded 8 MB really quick from the webseed,* but then it picked up on some mysterious peer from Estonia and switched focus. That's as intended, but it slowed my download speed from a few hundred kB/s to significantly less. :P That being said, my qBittorrent is still pulling in 500 kB parts from the webseed every... I don't know, half minute or so. It downloaded about 8 MB in the first few seconds while over the past 10 minutes it only downloaded an additional 5 MB. My guess is either qBittorrent is trying not to overload the webseed or the webseed isn't fond of the fact that torrent clients pull in parts here and there at random instead of properly downloading a file from beginning to end. Either way, that doesn't explain why it's not working for you.

Right, so I've got qBittorrent 3.1.10 built against Qt 4.8.6, Libtorrent 0.16.17.0 and Boost 1.55.0.

* Free hosting, yet it might even be able to host this site.  :insane:

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #46
I'm at 1.7 % now. Enough for this weekend. Will try more next weekend.

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #47
Yes, you picked some up from me after I got it from the webseed. :P

But I think there might be a bug in Libtorrent. Although I've downloaded 60 MB this session, 50 MB of it was apparently wasted(!), i.e., failed the consistency check. This obviously makes no sense because I'm sure that if I'd download the complete file using wget it'd be picked up just fine.

Here's a bugreport for what sounds like the same issue, and perhaps it offers the answer:
Quote
There is a known issue that if the web server doesn't support keep-alive, 16 kiB blocks that would require more than a single request won't work. libtorrent currently must request an entire block within the same connection. If the server closes the connection, the partial block that may have been downloaded is discarded (just like normal bittorrent peers).

The symptom you describe sound like what you would see with multi-file torrents and web server that don't support keep-alive.

I'm not sure how to check if the server supports keep-alive (probably something with wget or curl), but in any case I did put it on the server as many small files of a few MB.

  • ersi
  • [*][*][*][*][*]
Re: My Opera Backup
Reply #48
After an abrupt restart my qBittorrent totally fails to launch. Even reinstalling and wiping out the profile didn't help. Other Qt programs such as Qpdfview and Otter are unaffected. I am bad at debugging, so I simply uninstalled the thing. I'm pulling the torrent now with Aria2 which never failed me.

  • Frenzie
  • [*][*][*][*][*]
  • Administrator
Re: My Opera Backup
Reply #49
So the webseed is working well with this Aria2? Interesting.