Filed Plagiabot issues:

• https://github.com/valhallasw/plagiabot/issues/6
• https://github.com/valhallasw/plagiabot/issues/3#issuecomment-175243480

Final find: a missing comma in PageAssessments' extension.json.

From the vagrant git-update output:

-----------------------------------------------------------------
wiki
-----------------------------------------------------------------
wiki:  #!/usr/bin/env php
PHP Fatal error:  Uncaught exception 'Exception' with message '/vagrant/mediawiki/extensions/PageAssessments/extension.json is not a valid JSON file.' in /vagrant/mediawiki/includes/registration/ExtensionRegistry.php:187
Stack trace:
#0 /vagrant/mediawiki/includes/registration/ExtensionRegistry.php(137): ExtensionRegistry->readFromQueue(Array)
#1 /vagrant/mediawiki/includes/Setup.php(39): ExtensionRegistry->loadFromQueue()
#2 /vagrant/mediawiki/maintenance/doMaintenance.php(97): require_once('/vagrant/mediaw...')
#3 /vagrant/mediawiki/maintenance/update.php(214): require_once('/vagrant/mediaw...')
#4 /var/www/w/MWScript.php(89): require_once('/vagrant/mediaw...')
#5 {main}
  thrown in /vagrant/mediawiki/includes/registration/ExtensionRegistry.php on line 187
wiki:  
wiki:  Fatal error: Uncaught exception 'Exception' with message '/vagrant/mediawiki/extensions/PageAssessments/extension.json is not a valid JSON file.' in /vagrant/mediawiki/includes/registration/ExtensionRegistry.php on line 187
wiki:  
wiki:  Exception: /vagrant/mediawiki/extensions/PageAssessments/extension.json is not a valid JSON file. in /vagrant/mediawiki/includes/registration/ExtensionRegistry.php on line 187
wiki:  
wiki:  Call Stack:
wiki:      0.0002     232368   1. {main}() /var/www/w/MWScript.php:0
wiki:      0.0141     331224   2. require_once('/vagrant/mediawiki/maintenance/update.php') /var/www/w/MWScript.php:89
wiki:      0.0210     689840   3. require_once('/vagrant/mediawiki/maintenance/doMaintenance.php') /vagrant/mediawiki/maintenance/update.php:214
wiki:      0.3113    5323360   4. require_once('/vagrant/mediawiki/includes/Setup.php') /vagrant/mediawiki/maintenance/doMaintenance.php:97
wiki:      0.3158    5449272   5. ExtensionRegistry->loadFromQueue() /vagrant/mediawiki/includes/Setup.php:39
wiki:      0.3217    5738344   6. ExtensionRegistry->readFromQueue() /vagrant/mediawiki/includes/registration/ExtensionRegistry.php:137
wiki:

@bd808: More errors.

Previous work/background (as linked above): https://www.mediawiki.org/wiki/Archived_Pages

@Slowking4: Thanks for the report. I'm working on other Community Tech projects and am not able to add more features right now. However, if there's anyone who wants to submit a patch for this or is interested in taking up maintainership I can absolutely work with them.

Useful follow-up: improve documentation for ApiBase::getAllowedParams().

Similar command-line tool to identify probable deadlinks (pywikibot-based): https://www.mediawiki.org/wiki/Manual:Pywikibot/weblinkchecker.py

A proposal from @Legoktm: https://www.mediawiki.org/wiki/User:Legoktm/archive.txt

@Niharikakohli and I are discussing this at the #DevSummit16. Questions:

• Who has the most expertise and who should Community Tech talk with?
• What are the desired outcomes? Currently noted:
  • A way to get the false/true positive checks into the DB
  • A way to get turnitin to check a revision on demand
  • UI/UX improvements (extensionise?)
  • WikiProject integration (how? A fairly enwiki-specific usecase--how broadly requested is this)
  • API improvements
• What is the minimum viable product for this? What minimal changes will improve the flow/experience?

Working branch until I figure out appropriate dependencies/remotes in Gerrit: https://github.com/fhocutt/Assessments/tree/T119997-write-API

Pull request: https://github.com/earwig/copyvios/pull/24

<Earwig> fhocutt: this should probably be tweaked internally, but you should be able to (as an immediate fix) import earwigbot; earwigbot.bot.Bot("<path to the bot's directory>").wiki.add_site(lang="en", project="wikipedia")

Yup, sounds like a plan.

Wiki comment with updates: https://en.wikipedia.org/wiki/User_talk:Citation_bot#Update:_Security_fixes_and_code_cleanup

Re: the API:

Does the old database still exist?

If it does, it is the mysterious core file in the tool labs repository, but I wasn't able to log in with the old credentials.
Additionally, even if it is, it will be 2-3 years out of date and probably not useful.

How many blocking issues are there?

Unclear how many are blocking, but new issues have been reported recently (link)

Is there any sort of consensus on how good it needs to be to run automatically?

None mentioned; it is unclear whether it would need to have no open bugs, or whether a certain level of bugginess is acceptable.

What policies etc. would it need to comply with?

Standard enwiki bot policies; technically it wouldn't need to be reapproved, but consensus might demand this; unclear.

Who would maintain this/keep up to date with issues raised?

No one has volunteered as a maintainer, so there is not anyone right now.

Asked here:

• https://en.wikipedia.org/wiki/User_talk:Citation_bot#Questions:_Interest_in_and_prerequisites_for_starting_up_automated_citation_fixing
• https://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Citation_bot:_gauging_interest_in.2Fprerequisites_for_the_automated_citation_fixing_function

What are the current relevant policies and conventions?

The cite doi, cite pmid, etc. template subpage pattern (which the bot assumes) is now deprecated.

Awesome! Kaldari, could you use your admin privs to update this on enwiki?
The backend's live also.

The corresponding API is now available and up on citations on Labs.

@Earwig: thanks for the dev fixes! That's working now.

Could use a bit of refactoring, but works for this minimal case.

https://github.com/ms609/citation-bot/pull/102

Gadget is here for now: https://test.wikipedia.org/w/index.php?title=User:Fhocutt_%28WMF%29/common.js

That was it, thanks. Now I'm getting a KeyError in _get_search_engine
(stack trace below in case it has extra info)--it looks like I do need to
specify an engine and credentials for any search to work. What's the best
way to do that? Can I use the BOSS creds, or is there another alternative?

All right! Thanks for taking the time to reply here.

@Earwig: What's the best way for me to contribute to this? Is there a development repository, is there a good way to run it locally, should I get a copy set up on the community-tech-tools project? Also, thank you for the thorough documentation on Copyvios and EarwigBot. I ran into some issues but it's still much easier so far than some other tools I've worked with.

@Halfak: Sounds good. I look forward to working on it. And yes, that would have been very helpful, as informative and interesting as the bug-hunting was...

@Halfak: for a side project, we ended up improving WikiTalkParser. Some of it is better used as inspiration than re-used in a library, but it works now.

Merged and pulled up to labs repo: https://github.com/ms609/citation-bot/pull/101, https://github.com/fhocutt/citation-bot/pull/2 .

Merged and pulled onto Tool Labs.

Answer from Jonesey95:

The CS1 citation module (and its predecessor code) that renders {{cite journal}} and similar templates used to display "et al." when exactly nine authors were listed, for reasons too boring to go into. The module no longer does that; it displays all authors unless |display-authors=x is used (where x is a number less than the number of authors in the cite template). So |display-authors=9 is redundant when there are exactly nine authors listed.

Related branch/commit on github: https://github.com/fhocutt/citation-bot/commit/40968c75f4f3fe335cddc2fdd6cdbccbda268b4f

Perhaps the {{cite journal}} template on testwiki is out of date? https://test.wikipedia.org/w/index.php?title=User:Fhocutt_%28WMF%29/Citation_bot_test&oldid=244444 shows et al. still present, but looking at the same citation here https://en.wikipedia.org/w/index.php?title=Penicillium_pinophilum&type=revision&diff=682189717&oldid=682189517 shows all authors.

This looks easy, but code comments make me wonder whether this is a good change to be making. The code to insert display-authors=9 is a special case, inserted so that "et al. " would not hide only one author (from code comments: "Better display an author's name than 'et al' when the et al only hides 1 author!"). I am not sure whether the reasoning that led to this special case is still valid. Commented here to ask: https://en.wikipedia.org/wiki/User_talk:Citation_bot#.7Cdisplay-authors.3D9_no_longer_necessary_for_exactly_nine_authors

Related pull request for review: https://github.com/fhocutt/citation-bot/pull/2