XIVAPI is a toolkit that allows people managing FFXIV websites to keep their sites updated with in-game data. I've wanted to integrate the FFWiki to XIVAPI for a while now, but always have hit either technical snags or real-life difficulties. However, I think it should now be feasible. In fact, XIVAPI now has a client written in Python (which is the same language as Intangir Bot), XIVAPI's Discord login bug has been fixed, and I have a bit more free time over the next few months than I usually do. Also, Endwalker is rapidly approaching.
Currently, my only explicit plan for the bot is to write quest pages, but I could potentially do anything with it that is in the scope of XIVAPI. Cat (meow ∙ hunt) 19:38, 4 May 2021 (UTC)
Quests
I wrote a script for Intangir Bot that can automatically write quest pages using XIVAPI's client. See Project:Sandsea/On Thin Ice. In total the bot would write about four thousand pages. It's not quite ideal yet, as I need to figure out how to integrate quest banners into it.
Some questions before I go run this script:
- Most importantly, are we OK with this? I know when we started using bots just to fix links there was some backlash from some users who felt it was akin to sockpuppeting -- now that bots can write pages this is a significant expansion in the scope of what they can do.
- Is this page format OK? I based it on Chasing Shadows. Do we want the bot to create empty sections like "Walkthrough" for humans to fill in?
- Where should the item links point to? Specifically should they point to the relevant item pages? I think there was some talk about breaking up item pages because they are too long, and I don't want to generate thousands of pages and then learn that the links on those pages will die in a few weeks.
- Should bot-generated pages have mboxes saying they are bot-generated so that humans can go in and fill in the little things that bots get subtely wrong? Cat (meow ∙ hunt) 19:38, 4 May 2021 (UTC)
- Having bots write the pages and allowing users to go through and make necessary adjustments will save a gargantuan amount of work which I am more than ok with.
- Once I get items integrated with {{icon}}, my plan was to just have them link to their name (and try to identify disambigs and the like), which atm will just mean a ton of red links everywhere. Once a decision is made on how to do item pages, the links for everything will be ready to go already, just redirects will need to be made if necessary.
- Having a bot-generated box, that can also add them to a Category:Bot-generated pages or something, I think would be nice so we know which pages might need to be touched up, and we can remove them as we go.
- When it comes time for new quest pages, like say when 6.0 hits, would we just run the bot again for the new quests? I can't imagine we'd have it rebuild every quest page every patch and overwrite user contributions, so users would have to keep an eye out for updates to old quests.
- Also, how will the bot handle multiple quests of the same name, such as Close to Home? While I built that as a single page, it actually represents eight different quests in the backend.
- Sidenote, I'm impressed the bot built the script section as well as it did. Adonzo (talk) 19:59, 4 May 2021 (UTC)
- ETA: Also the infobox links to items and duties and be switched to their {{icon}} links as well. I actually got distracted from building the quest pages by uploading item icons specifically so I could integrate them with the infoboxes lol. Adonzo (talk) 20:12, 4 May 2021 (UTC)
I think mboxes are fine, saying "a human hasn't edited this page yet". And I'm perfectly fine with bots taking over from mind-numbing, repetitive tasks—that's exactly what they're there for.--
Technobliterator TC 20:28, 4 May 2021 (UTC)
- just redirects will need to be made if necessary.
It sounds like the bot should help generate redirects or something, which I guess could be the next project after this Quest project?
- when 6.0 hits, would we just run the bot again for the new quests?
Yes. In-game every quest is identified by its ID, which is an integer between 65336 and 70000 or something. The bot takes in an integer range and generates pages for all of the quests whose ID is in the given range, so there's no risk of the bot overwriting previously written quests. It might be a problem if SE reworks a bunch of quests again (as happened in 5.3).
- how will the bot handle multiple quests of the same name
If the bot can't create a page because there's a page at that title already, it will instead write to its own userspace using the quest ID, and I'll just figure out how to merge it into mainspace afterwards.
- Also the infobox links to items and duties and be switched
Sure, is the syntax just going to be {{icon|FFXIV|item|Heaven's Arrow Materia IX}} and {{icon|FFXIV|duty|Eden's Flying Circus (Ultimate)}}?
- I'm impressed the bot built the script section as well as it did.
The way that FFXIV stores text is some jank XML-like language that I've partially managed to figure out. So there are probably still lots of bugs in how the bot builds script sections (and it certainly can't provide context to each scene in the same way that a human-written script section could, hence the tidbit of Ryne saying, "It's life!" with no context lol), but for the 10-20 pages that I tested it on, including On Thin Ice, I wasn't able to find any bugs.
- I think mboxes are fine, saying "a human hasn't edited this page yet".
I propose Template:Bot-generated. Cat (meow ∙ hunt) 22:12, 4 May 2021 (UTC)
Okay, bot work is done. Some things that need to be done by hand:
- Lots of quests are missing or in the wrong order in Module:XIVAPI/data due to various issues with the bot, quests getting renamed, quests being stored in weird places in the game data, etc.
- XIVAPI doesn't store how much EXP a quest gives or what quests it unlocks, so that needs to be added.
- XIVAPI is frequently wrong about what patch or expansion a quest was added to, so I just rejected that altogether. So that needs to be added.
- Banners are often stored superfluously on XIVAPI, so I rejected that, so that needs to be added.
- Context for dialogue, and dialogue options (stored in the talk page!) needs to be added. Also dialogue formatting is broken sometimes (because, again, that part of the game is written in some horrible quasi-XML, which does not actually meet the syntax specifications of XML).
- Screenshots of the quest, and walkthroughs of technical parts of the quest, need to be added.
- The mainspace talk pages at Category:Quests needing attention need dialogue options.
- The projectspace talk pages at Category:Quests needing attention need to be merged into mainspace. I'm working on this, there's about 100 so it'll take some days.
- The quest list pages in Template:navbox quests FFXIV need to be written. Not sure how they should be formatted.
A prototype page is Every Little Thing She Does Is Magitek. The code for the Intangir Bot script is at Final Fantasy Wiki:XIVAPI/Code/generate quests.py.
Special thanks to Adonzo, who did a ton of manual work, especially with respect to Module:Icon, that was the reason why this went so smoothly. (It goes without saying that XIVAPI and Lulu's Tools also were essential.) Because my time is limited and I figure we should try to automate as much as we can before Endwalker, I figure once the projectspace pages in Category:Quests needing attention are sorted out and the quest list pages are laid out, I should probably move on to the next thing. But I'm not sure what that should be.
Perhaps the next thing should be items -- but I know Adonzo has done a ton of manual work here and I'm not sure how the bot should pick up from his stuff, or if that would even be helpful. Thoughts? Cat (meow ∙ hunt) 03:21, 16 May 2021 (UTC)
- I would not at all be offended by a bot taking over and potentially overwriting the item pages. They really just can't be feasibly maintained manually. Only really I got so much done as I did due to having a lot more free time than usual the past year because of the pandemic.
- Also, with the impending stat squish of Endwalker, they will all need to be redone anyways. Adonzo (talk) 03:48, 16 May 2021 (UTC)