Anyone with web scraping experience want to help archive old RTS scripts?

[hugecat]

@PHO3NIX-E I like xrobin’s suggestions, but let us know how much work it is. If there’s a faster method that still gets us like 90% of the way, we could potentially do that as well.

Also are you scraping any of the comments? If not, how do we want to handle that in case there are any broken links in the original post?

In #script-creator-portfolios, I can also create a pinned post explaining that past scripts have been scraped from RTS and we will remove anyone’s content removed if the author requests it.

[xrobin]

I just got an idea that might sort of help. We could scrape the RTS post url along with everything else, and then include that link in the list, so that way if someone discovered a broken link, they could very easily pop over to RTS and see if there’s a fixed link in the replies. What do you think?

[hugecat]

Maybe you could scrape all comments, but only keep comments that have a link/attachment in them. Then tack them onto the original post’s content?

[PHO3NIX-E]

I’ll scrape every topic for every hyperlink, then testing them afterwards is inevitable i guess.

[hugecat]

If there’s a lot of manual labor involved in part of this process, we could try to enlist other users to help crowdsource the work.

[xrobin]

If that’s possible, it would be better than my idea since my idea would end up with a lot of broken links if RTS did go down.

[Realcumber]

@xrobin

I’d tend to think it’s not as necessary to scrape the paid section, since those are going to be browsable on SLR, RS, and CzechVR. I guess it might be nice if someone here is searching for their favorite actress or studio and this could be a one-stop source for searching all scripts, but I suspect many pro scripters may get around to their own portfolios at some point.

Good discussion here guys, I am curious to see how this turns out, very important that you have that pinned statement stating creators can request takedowns of their content if not reached out to.

A one stop spot for paid and free is huge actually, where people can look to see if there favorite scene has been scripted (paid or free)

As long as there arent commercial paid scripts included then should be no problem, and if this could consolidate listing of all paid scripters work into #script-creator-portfolios , that would be welcome as well

[hugecat]

Best practices for scripters migrating their old scripts to EroScripts?

When I joined the RTS forums, the Launch didn’t exist. The RealTouch had recently died due to patent trolling, and finding scripts for the Vorze Cyclone was most easily done there. I, too, was confused by the paid/free script section paradigm, but it’s absolutely as I described. The free/paid description refers to how you can access the video, and has nothing to do with the script. Before the SyncArmy stuff started, nobody charged for scripts. They just posted them to the forum.

Best practices for scripters migrating their old scripts to EroScripts?

Good to know, that’s interesting. So after scraping, someone will need to go through the paid scripts and manually change the ones that are actually free scripts.

This is implying that the paid funscript and paid vr funscript section would also need to be scraped then?

[xrobin]

If possible, I think it would be good to scrape the paid section as well in light of both the news that there’s a lot of free scripts in there, and also Realcumber’s words above encouraging us to go ahead and migrate the paid scripts too.