Possible import of some EAv2 threads

[Neutrum260725]

So that everyone interested knows what this is about, here's a screenshot of a thread from EAv2.

Neutrum260725

[Neutrum260725]

As far as I know, fhunter's script was designed to allow users to copy the stories completely and download them as a zip file. That's exactly what I did back then, but I also started copying the archived threads from the actual Eunuch Archive to my PC. Since that was quite time-consuming, I couldn't copy everything. My copies are genuine EAv2 threads, which can only be read individually in each browser, just like the original. The missing avatars are probably due to the need to save storage space, and new posts are naturally not possible. The total size of the archive data alone is around 800 MB, significantly more than the data contained in the stories. I've attached the "Blogs & Life Stories" section as a zip file so that anyone with access can see for themselves. Other sections are probably much larger, so I've included a smaller one as an attachment for now.
neutrum260725

[WheelyFixed]

As far as I know, fhunter's script was designed to allow users to copy the stories completely and download them as a zip file. That's exactly what I did back then, but I also started copying the archived threads from the actual Eunuch Archive to my PC. Since that was quite time-consuming, I couldn't copy everything. My copies are genuine EAv2 threads, which can only be read individually in each browser, just like the original. The missing avatars are probably due to the need to save storage space, and new posts are naturally not possible. The total size of the archive data alone is around 800 MB, significantly more than the data contained in the stories. I've attached the "Blogs & Life Stories" section as a zip file so that anyone with access can see for themselves. Other sections are probably much larger, so I've included a smaller one as an attachment for now.
neutrum260725

I have the story archive, and a zip of the forum that someone made and got something similar to what you posted in that screenshot, where they didn't show what parts of the the messages were quotes from earlier ones in the thread....

I don't know if you missed attaching the attachment, or if it doesn't show up here for some reason, but I'm not seeing it...

WheelyFixed

[dandelion]

It'll be nice to see your version of the scrape. fhunter ran a scrape for the forum too. But quotes are included as plain text along with post contents, making it somewhat difficult to read.

I don't see your zip file unfortunately.

As far as I know, fhunter's script was designed to allow users to copy the stories completely and download them as a zip file. That's exactly what I did back then, but I also started copying the archived threads from the actual Eunuch Archive to my PC. Since that was quite time-consuming, I couldn't copy everything. My copies are genuine EAv2 threads, which can only be read individually in each browser, just like the original. The missing avatars are probably due to the need to save storage space, and new posts are naturally not possible. The total size of the archive data alone is around 800 MB, significantly more than the data contained in the stories. I've attached the "Blogs & Life Stories" section as a zip file so that anyone with access can see for themselves. Other sections are probably much larger, so I've included a smaller one as an attachment for now.
neutrum260725

[dandelion]

I tinkered with it a bit, and made some pretty good progress on a test phpBB instance on my private server. See screenshots below.

Since the scrape I worked with did not separate quotes from replies, I had to programmatically look for repeated paragraphs within each reply and look up the original author. It seems to have worked quite well, but we should perhaps do more in-depths double checking.

Things I need:
1. @fusion or @fhunter: Can you replicate the site onto a separate VM or container that I can test with? I'll need these access for the test site
1.1 Admin access to phpBB
1.2 SFTP
1.3 SSH, not sure if I need root because I did everything on my test server with root access
1.4 Access to VM's snapshot so I can revert back to the pre-import state and rerun as I refine my script
2. Some volunteers to help check for completeness, errors, and bugs with quote identification
3. Once the technical parts are done, we can discuss where and how to integrate old threads into the forum

[waj1993]

I'll volunteer for reviewing for issues, but i am not schooled in how to fix so feedback is all i can do.

[WheelyFixed]

This looks really good! I think it is definite progress!

WheelyFixed

[dandelion]

@Neutrum260725 and @waj1993 please check your PM

[Evolve]

I'm able to access the initial link on both pdf attachments, but if I click on any of the threads within the page that pops up, I get the following:
------
Hmmm… can't reach this page
192.168.12.24 took too long to respond
Try:

Search the web for 192 168 12 24
Checking the connection
Checking the proxy and the firewall
ERR_CONNECTION_TIMED_OUT

[WheelyFixed]

I'm able to access the initial link on both pdf attachments, but if I click on any of the threads within the page that pops up, I get the following:
------
Hmmm… can't reach this page
192.168.12.24 took too long to respond
Try:

Search the web for 192 168 12 24
Checking the connection
Checking the proxy and the firewall
ERR_CONNECTION_TIMED_OUT

I think that's because links are to pages on Dandelion's home server which doesn't allow outside access. I was getting the same thing.... He is making good progress so far - I've also been talking with him a good bit about it in email... One of the things that will help is to get that list of things he mentions in the post to get a better test and development environment...

WheelyFixed

[dandelion]

Yea that was an internal IP address of my home network. Imagine my home network is a hotel. That address tells housekeeping what floor and what room to knock on. Your home network most likely doesn't have that floor/room, and even if there was, housekeeping staff wouldn't find what they expected to find at my hotel.

I did reply to your PM.

I'm able to access the initial link on both pdf attachments, but if I click on any of the threads within the page that pops up, I get the following:
------
Hmmm… can't reach this page
192.168.12.24 took too long to respond
Try:

Search the web for 192 168 12 24
Checking the connection
Checking the proxy and the firewall
ERR_CONNECTION_TIMED_OUT

I think that's because links are to pages on Dandelion's home server which doesn't allow outside access. I was getting the same thing.... He is making good progress so far - I've also been talking with him a good bit about it in email... One of the things that will help is to get that list of things he mentions in the post to get a better test and development environment...

WheelyFixed

[WheelyFixed]

If I'm remembering correctly that 192.168.x.x range of addresses is supposed to be reserved for private / local networks as part of the addressing standard, sort of like 127.0.0.1 is always 'localhost' - I remember a User Friendly comic from years back where the admin was in chat with a bunch of script-kiddies who were threatening to shut him down and p0wn his system - he told them that was his I.P. address and the next pane was a collection of clicks as the kiddies made their own machines go 'poof'...

WheelyFixed

[waj1993]

Been going through alot of threads and haven't come across any glaring issues. They all read fine to me.

[Neutrum260725]

I didn't know that fhunter wrote another script to save the EAv2 threads. So it seems to be easier to bring old threads into the new board. If some of the old threads are missing, let me know, so I can look if I have them on my NAS and can send them to you.
Neutrum260725

[dandelion]

I didn't know that fhunter wrote another script to save the EAv2 threads. So it seems to be easier to bring old threads into the new board. If some of the old threads are missing, let me know, so I can look if I have them on my NAS and can send them to you.
Neutrum260725

The only thread that I know are missing is the Dec 2024 new story thread. It's hard to find out what I don't have LOL.

[Neutrum260725]

Sorry, but stories I didn't save.

[dandelion]

Sorry, but stories I didn't save.

Not the stories themselves. Paolo post a New Story thread to the forum every month.

[fhunter]

If I'm remembering correctly that 192.168.x.x range of addresses is supposed to be reserved for private / local networks as part of the addressing standard, sort of like 127.0.0.1 is always 'localhost' - I remember a User Friendly comic from years back where the admin was in chat with a bunch of script-kiddies who were threatening to shut him down and p0wn his system - he told them that was his I.P. address and the next pane was a collection of clicks as the kiddies made their own machines go 'poof'...

WheelyFixed

https://pbs.twimg.com/media/Dc0nIu1X0AA ... name=large

(sorry, that is an oldish meme:
"block your own anus, https://en.wikipedia.org/wiki/Roskomnadzor"
"should I? may be this is the way?"
"proceed with blocking of 127.0.0.1? Y/N"

(this was after 127.0.0.1 ended up on the official russian blocklist)

[fhunter]

I didn't know that fhunter wrote another script to save the EAv2 threads. So it seems to be easier to bring old threads into the new board. If some of the old threads are missing, let me know, so I can look if I have them on my NAS and can send them to you.
Neutrum260725

I didn't write the script to load the threads. I used https://en.wikipedia.org/wiki/HTTrack software for that. (Stories on another hand resisted my attempts).

I may have a look, I may have partial copy of the full forum. Need to check that, though. It kept failing to download. That may also include some of the avatars too. Do not get your hopes up, though.

[fusion]

One issue with avatars is phpBB (EAv3) and vBulletin (EAv2) use very different formats so even if the old ones are found they may not be able to used.

Just in case anyone asks the reason EAv3 is on phpBB is vBulletin is a paid product hence why EAv2 was stuck on a very old version from when it use to be free.

[WheelyFixed]

Actually EAv2 did pay a certain amount even for the version of vBulletin it was running on... I think the reason they were stuck on the version they used was more because of the challenges of upgrading the existing message base to a newer version (Sort of the same problem we are having now...)

I don't know how vBulletin stored the avatars, but I know that both it and phpBB basically use uploaded .jpeg or a couple of other standard image formats in a specified size, which may need adjusting (trivial in most image software). I would have to think that if vBulletin changed them to some other format, it should be possible to change them back to .jpegs and re-import them to phpBB, but regardless, for now it is not something we need to deal with at this point.

Also worth noting, as it was EAv2 policy to delete accounts after a year of no activity, which also eliminated the avatars of those accounts, there aren't that many that actually would have had avatars...

ART

[fusion]

Fair points.

I was hoping to get to a more open platform but have now decided I will just create the plugins for this site myself so then everything is built in house.

[WheelyFixed]

Sounds good.... I hope that you will be willing to share any plugins you create that aren't EA specific back into the phpBBS ecosystem... (Speaking as a Free Software advocate - I'm a long time FSF member and know Stallman reasonably well...)

WheelyFixed

[Mellytoninn]

If it would be possible for you to upload the scrape as an attachment or possibly onto MediaFire or any file sharing service, it would mean the world to me. I've tried copying as much info off internet archives snapshots as possible, but to no avail. It's all gone...

[Mellytoninn]

I didn't know that fhunter wrote another script to save the EAv2 threads. So it seems to be easier to bring old threads into the new board. If some of the old threads are missing, let me know, so I can look if I have them on my NAS and can send them to you.
Neutrum260725

Chemical Injections thread (the alcohol and salts one) has so much info, it had like 170 pages If I recall correctly. That's a massive need for me tbhon