Wikipedia:Bots/Requests for approval/TolBot

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at Wikipedia:Bots/Noticeboard. The result of the discussion was

Approved.

TolBot

Operator: Tol (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 21:48, Friday, April 16, 2021 (UTC)

Automatic, Supervised, or Manual: supervised

Programming language(s): Python

Source code available: User:TolBot/Task 1#Source

Function overview: The bot would automatically update Template:COVID-19 vaccination data from the [CSV data file] from Our World in Data (the source already used for this template).

Links to relevant discussions (where appropriate): None yet

Edit period(s): Daily

Estimated number of pages affected: 1

Exclusion compliant (Yes/No): No

Already has a bot flag (Yes/No): No

Function details: The bot:

Downloads the CSV file.
Processes it in accordance with the instructions at the template documentation.
Generates wikitext in a similar fashion to the spreadsheet formula in the documentation.
Logs in to its account.
Downloads the page's wikitext.
Uses a regular expression to replace the data in between the comments with the updated data.
Makes an edit with the new wikitext.

Discussion

Very cool! The table layout has been pretty stable for a while. Only recent changes involved excluding specific regions. If we need to make minor adjustments in the future, is the Python code fairly easy to update? Bom4446 has been handling the daily updates so I'm sure they would welcome the break. - Wikmoz (talk) 07:07, 17 April 2021 (UTC)[reply]

Approved for trial (14 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete. Primefac (talk) 10:25, 17 April 2021 (UTC)[reply]

@Primefac: Should I do this on the bot account or on my main account? If it's on the bot account, I'll need a confirmed flag. (Sorry, this is my first time doing this!) It seems that I'll also need a bot flag or IP-block exemption flag, as the IP from Google Cloud that I'm using is blocked. Tol | Talk | Contribs (formerly Twassman) 15:16, 17 April 2021 (UTC)[reply]

@Primefac: I'm currently running the script to do everything but actually edit the article as the cloud IP is blocked, then editing manually (copy/paste) on my main account. Please advise — should I continue this or is there some way I can get either an IP-block exemption (on my main account) or confirmed and bot groups (on my bot account)? Tol | Talk | Contribs 15:37, 19 April 2021 (UTC)[reply]

Trial complete. Diffs are here. No problems other than the sorting issue, which is fixed. Tol | Talk | Contribs 17:12, 3 May 2021 (UTC)[reply]

Update: I have added some extra code and cleaned up generate() so that it handles cases with only people fully vaccinated and neither people vaccinated (at least once) nor total vaccinations; this is because Oman currently only has data on people fully vaccinated. Tol | Talk | Contribs 01:27, 9 May 2021 (UTC)[reply]

I can update the code as needed; all foreseeable updates will be to only one function (generating the wikitext from the processed table). I'll follow the talk page, but if you'd like me to change anything, please send me a ping/mesage! Tol | Talk | Contribs 15:27, 17 April 2021 (UTC)[reply]

Sounds good. It looks like there's one minor issue in the current build. Several (non-COFA) countries with high vaccine counts are appearing at the bottom of the list, including Egypt, Belarus, and Pakistan. - Wikmoz (talk) 05:16, 18 April 2021 (UTC)[reply]

@Wikmoz: Thanks for catching this! The issue seems to be that it sorts by total_vaccinations, but El Salvador, Belarus, Pakistan, and Egypt do not have total_vaccinations, only people_vaccinated. I'll set it up so that it checks for blank total_vaccinations and substitutes people_vaccinated instead. Tol | Talk | Contribs 18:13, 18 April 2021 (UTC)[reply]

@Tol This is indeed very cool! Thank you very much! Although I will continue updating the daily updates whenever possible. OWID usually updates their data around noon daily (CEST). The time is very convenient for me and also I can update the table as soon as new data becomes available. Bom4446 (talk) 14:11, 19 April 2021 (UTC)[reply]

@Bom4446: Thank you, and you are welcome! I'll see if I can run it at around 16:00 UTC (18:00 CEST) during the trial period. Tol | Talk | Contribs 23:26, 19 April 2021 (UTC)[reply]

Hi @Bom4446, could you please refrain from updating the table so that the bot can do it? Thanks, Tol | Talk | Contribs 15:34, 20 April 2021 (UTC)[reply]

I think the bot has done good work during the trail. Will it run again?Nico (talk) 05:44, 7 May 2021 (UTC)[reply]

@Nicob1984: The bot is currently awaiting approval. If it is approved (and it probably will be), it will run indefinitely. Tol | Talk | Contribs 16:36, 7 May 2021 (UTC)[reply]

Approved for extended trial (7 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete. Sorry for the delay, just want to make sure all of the issues listed above have been sorted. Primefac (talk) 15:18, 10 May 2021 (UTC)[reply]

Trial complete. Diffs are here. There was a fairly large issue here, because OWID (the source of the data) added additional rows for income groups, and the bot mistakenly copied those into the template. I added those to the row blacklist (which also contains things like continents and UK member states). I do not know if this will happen again, and am unsure if a better alternative would be to have a row whitelist, so that a row has to match some country name to be added (the problem is that if OWID adds an actual country, the data would not reflect this, so I would have to periodically check for new countries). Tol | Talk | Contribs 19:14, 16 May 2021 (UTC)[reply]

Great job! I'd suggest sticking with the blacklist since incorrect additions are easier to spot than omissions and easy to temporarily clean up manually. We can add a note to the template instructions for editors to ping you on the template talk page if there's an issue. In general, it might also be a cool feature to provide editors with a way to easily disable the bot from within the template code. Does it currently look for a speicific comment tag that we can modify to temporarily disable the bot if there's an issue and you're unavailable? - Wikmoz (talk) 02:04, 18 May 2021 (UTC)[reply]

@Wikmoz: No; it currently does not. I will set it up to look for a comment on the template and not edit if it is present. Tol | Talk | Contribs 04:31, 18 May 2021 (UTC)[reply]

Approved. Any remaining issues seem to either be fixed or easily fixable, so I suppose there's not much point in holding off on this. Primefac (talk) 13:57, 25 May 2021 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at Wikipedia:Bots/Noticeboard.