This is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.
Fixed in Portal:Earth sciences/Selected article/2. The paragraph selection option is a named parameter, so the template needs paragraphs=1-2 rather then just 1-2. I'll see if I can make the documentation clearer. Certes (talk) 17:41, 27 April 2018 (UTC)
I ran into a curious problem with Portal:Earth sciences/Selected article/8. In a previous revision of Climate of India, there was a break in the middle of the caption of the third figure, and the transclusion displayed the figure as well as a badly formatted chunk of the caption in the lead. I didn't request a figure in the template. After I fixed the caption (this revision), it worked fine. RockMagnetist(talk) 23:36, 27 April 2018 (UTC)
Thanks for the report. File: and Image: can contain nested wikilinks and I've been a bit lazy in parsing them (which speeds the module up). I'm assuming that they take up the rest of the line, which is usually but not always the case. I should probably fix that soon, but it's good that you've patched things up for now. Certes (talk) 00:11, 28 April 2018 (UTC)
Yes, it's not urgent but I'll let you know if there are any more issues. RockMagnetist(talk) 02:05, 28 April 2018 (UTC)
"Bad argument #1 to 'title.new' (number or string expected, got nil)"
I keep seeing the error above (clicking on it links to line 16) on some of the selections at User:Waggers/sandpit/portal selection random. (If you don't see the error at first, click "purge" to select a different article). Annoyingly the error doesn't always occur - but seems to more often than not.
It seems to be indicating that the pagename variable is not populated, yet the same page name is fed to {{more about}}, which always seems to display correctly, and the supposed lack of a pagename is not captured and handled by line 7, which looks like it's testing for pagename=nil. Am I doing something wrong? WaggersTALK 13:04, 3 May 2018 (UTC)
Thanks for the report. Whisper it quietly, but function lead (for Transclude lead excerpt) and function random (for Transclude random excerpt) are identical. They pick one of their unnamed parameters at random and treat it as an article title. In this case you're passing three unnamed parameters to lead. If your random number is 3 then these are article3, paragraphs3 and leadimages3, which evaluate to "Jane Austen", "" and "" respectively. One time in three you show part of Jane Austen; two times in three you try to show the article "" and get an error message. I can change the module so lead ignores all unnamed parameters except the first (making it differ from random), and I'll see if I can put out a more helpful error message. I should probably also make random ignore blank parameters and possibly try a different random number if it hits a missing article. Certes (talk) 13:35, 3 May 2018 (UTC)
I see, thanks. My bad - I've now named those parameters in the call to {{Transclude lead excerpt}} (they should be |paragraphs= and |files= instead of unnamed. Thanks v much for the quick response. WaggersTALK 15:06, 3 May 2018 (UTC)
Module changed as above. Choosing a random excerpt from "cat" and a bunch of invalid titles now always tells you what a cat is. I've also added errors= to show conventional Lua error messages rather than a blank excerpt. Certes (talk) 16:30, 3 May 2018 (UTC)
I would like to request a small bug-fix if possible. The Transclude Lead Excerpt template leaves out the Template:Rp. An example below (note the superscript “:11” after “integral part of the airline's business”):
Biman Bangladesh Airlines (Bengali: বিমান বাংলাদেশ এয়ারলাইন্স) commonly known as Biman (pronounced /biːmɑːn/ [bee-mah-n]) is the national flag carrier of Bangladesh. With its main hub at Hazrat Shahjalal International Airport in Dhaka, the airline also operates flights from its secondary hubs at Shah Amanat International Airport in Chittagong and as well as Osmani International Airport in Sylhet. The airline provides international passenger and cargo services to multiple destinations and has air service agreements in 42 countries. The headquarters of the airline, Balaka Bhaban, is located in Kurmitola, in the northern part of Dhaka. Annual Hajj flights, transporting tourists, migrants, and non-resident Bangladeshi workers and the activities of its subsidiaries form an integral part of the corporate business of the airline. Bangladesh's air transport sector, which is experiencing an 8% annual growth rate thanks to a large number of outbound tourists, domestic tourists, and non-resident Bangladeshi travelers, is very competitive with stiff competition among a number of private Bangladeshi airlines as well as Biman.
Created in February 1972, Biman enjoyed an internal monopoly in the aviation industry of Bangladesh for 24 years, until 1996. In the decades following its founding, the airline expanded its fleet and destinations but it was adversely affected by corruption and mismanagement. At its peak, Biman operated flights to 29 international destinations, extending from New York City in the west to Tokyo in the east. The airline was wholly owned and managed by the government of Bangladesh until 23 July 2007, when it was transformed into the country's largest public limited company by the Caretaker Government of Bangladesh. Since becoming a public limited company in 2007, the airline has reduced staff and begun to modernize its fleet. The airline had signed a deal with Boeing to buy ten new aircraft along with options for ten more in 2008. After getting delivery of the new planes, Biman expanded its destinations and increased in-flight amenities, especially onboard Internet, WiFi, mobile telephony and live TV streams.
During his visit to Dhaka in September 2023, French President Emmanuel Macron, together with Bangladesh Prime Minister Sheikh Hasina, announced a large order of ten Airbus A350 aircraft for Biman. The Airbus order consists of the purchase by Biman Bangladesh Airlines of two A350F cargo aircraft and eight A350 passenger aircraft. Delivery of the aircraft will be done in stages with two passenger A350 aircraft joining the Biman fleet first. Biman Bangladesh Airlines is certified as safe to fly in Europe by the European Union Aviation Safety Agency. In addition, Biman has also successfully passed the IATA Operational Safety Audit and since then, the airline has resumed flights to some of its previous destinations in Asia and Europe. In recent times, Biman Bangladesh Airlines has seen a marked improvement in punctuality, as well as in on-time flight performance, under its new management team. (Full article...)
Default for parameter files appears to be no files rather than all files as stated on the description page. · · · Peter (Southwood)(talk): 04:51, 7 May 2018 (UTC)
I see that the description varies in different places. I will fix it to comply with reality. · · · Peter (Southwood)(talk): 04:54, 7 May 2018 (UTC)
issue
On the CKB Wikipedia nothing seems to be happening when I use the Transclude lead excerpt template. It doesn't display any text from the main article. Do you think this is because it doesn't support RTL languages or I'm doing something wrong? I used it here.--◂ épineⓋ 01:01, 2 May 2018 (UTC)
I don't know! To be honest, I wrote the alpha draft of the module with English Wikipedia in mind and didn't think very hard about internationalisation at this early stage. I am quite surprised that the module has been taken up so widely and so quickly. Please can you provide links to the module and the template on CKB Wikipedia? I've not seen Sorani before and I can't actually work out how Excerpt was being used in the Evanescence portal. Arabic Wikipedia seems to be using the module ar:وحدة:Excerpt with RTL text in ar:بوابة:تشريح. Hope that helps, Certes (talk) 01:29, 2 May 2018 (UTC)
@Épine: It works on Kurdish wiki. Look here brother. I am happy to help.-مصعب (talk) 17:08, 13 May 2018 (UTC)
@مصعب: I see, it seems I wasn't using the template quite correctly, however, in my defense, the documentation for the template is not really clear. That's understandable since it's still being worked on.--◂ épineⓋ 19:54, 13 May 2018 (UTC)
@Épine: Please feel free to suggest what is unclear in the documentation, or just go ahead and improve it. But as you say, although several portals are already using the templates, this is an alpha module and is still being developed. The module was specifically aimed at a current initiative to improve English portals, but if other wikis are using the software then perhaps we should think more seriously about making it work better in other languages, including RTL text. Certes (talk) 20:46, 13 May 2018 (UTC)
You should definitely keep the other wikis in mind. We rely on these types of modules more than you think.--◂ épineⓋ 21:15, 13 May 2018 (UTC)
The transclude lead excerpt template does not remove the {{sfnm}} template, which is a template for multiple refrences in one note. Dreamy Jazz (talk) 15:07, 20 May 2018 (UTC)
@Dreamy Jazz: I've now attempted to fix that problem, but I can't find an example of Sfnm being used in the lead. Please can you check the page which was broken, and confirm whether it has improved? Thanks, Certes (talk) 15:28, 20 May 2018 (UTC)
@Certes: I can no longer see them on Portal:Argentina, which is the place I noticed the problem. It seems to be fixed, thanks Dreamy Jazz (talk) 16:02, 20 May 2018 (UTC)
I have an instance of failing to return to unitalicised text on text starting The '''SS ''Vienna''''' was built in 1873 . Text starting with "Vienna" and continuing through "was built..." until next instance of '' all in italics. Possibly misinterpreting the '''''. See below
The SS Vienna was built in 1873 during the era when steamers were built with sail rigging. She had a 19 year career marked with maritime incidents including sinking when she was just three years old. She sank for her final time in fair weather in Whitefish Bay in Lake Superior after she received a mortal blow when she was inexplicably rammed by the steamer Nipigon. Although there were no deaths when the Vienna sank for the last time, more than 100 years later her wreck claimed the lives of 4 scuba divers, the most of all the wrecks in the Whitefish Point Underwater Preserve that now protects her as part of an underwater museum. Her wreck was stripped of artifacts that resulted in the Michigan Department of Natural Resources seizing her artifacts in a raid on the Great Lakes Shipwreck Museum in 1992. Her artifacts are now on display in this museum as loan from the State of Michigan. (Full article...)
@Pbsouthwood:Fixed. Thanks for the report. The nested apostrophe parser now handles partially italic titles in bold, such as '''SS ''Vienna''''' and '''''Billboard'' 200'''. I've also tweaked the image parsing so that it displays the ship image in preference to the U.S. flag. Certes (talk) 12:20, 22 May 2018 (UTC)
Thank you for fixing it. The least we can do as users is try to give you accurate bug reports. It is getting better than it was, so a net gain for the project. · · · Peter (Southwood)(talk): 12:37, 22 May 2018 (UTC)
Many article leads only have an image in the infobox. These would be desirable in the transcluded leads for portal selected articles if parameter files is set. It would also be better to specify image size in the template than to transclude the size used in the article, which is often much too large for the portal frame. · · · Peter (Southwood)(talk): 04:38, 7 May 2018 (UTC)
I agree that these are good ideas. In theory, an infobox which uses full image syntax per {{Infobox/doc#Illustration image}} should be easy to parse. In practice, each project has invented its own syntax such as {{Infobox film/doc#Image}} (which also has optional alt= and image_size= parameters). I'll have a look at it but it's probably better to leave the feature out than to implement it unreliably.
I could add an image size (or a list of sizes for multiple images?), but would it be better simply to use the [[File:...]] syntax outside the template? If the article changes to use a different image, we may well want to change the size in the excerpt too, rather than have it automatically apply the sizing chosen for a different image. Certes (talk) 09:35, 7 May 2018 (UTC)
Possibly my ignorance will show here, but how difficult would it be to extract just the file name and the caption, and combine them as a default size thumbnail at the top of the excerpt? · · · Peter (Southwood)(talk): 18:03, 7 May 2018 (UTC)
It seems to be almost getting the image from the infobox at James Talacek here:
[[File:NEEMO 14 James Talacek wearing LifeShirt.jpg|Talacek inside Aquarius during the |thumb| James Raymond Talacek is an American professional aquanaut with the University of North Carolina Wilmington (UNCW). He serves as Oceanographic Field Operations Manager at Aquarius Reef Base, the world's only undersea research laboratory. Read more...
It seems to have hit a problem with a link in the caption. · · · Peter (Southwood)(talk): 11:19, 22 May 2018 (UTC)
Yes: it misreads the | within the piped link as ending the infobox's caption= parameter. I'll see what I can do, but a bulletproof wikitext parser would take years to write and shouldn't really go in such a specialised module. Certes (talk) 12:23, 22 May 2018 (UTC)
Whatever you can do is appreciated. Cheers, · · · Peter (Southwood)(talk): 12:33, 22 May 2018 (UTC)
Fixed, though that bit of code is more complex than I'd have liked. I see a separate bug with his name not being wikilinked. I'll take a look at that next. Certes (talk) 14:46, 22 May 2018 (UTC)
Name linking bug also fixed. (The long image caption made it appear that the bold text occurred too late in the article to be a synonym for its title.) Certes (talk) 14:59, 22 May 2018 (UTC)
Idea
It would be really helpful if the module could include the infobox picture in the excerpt instead of ignoring it completely.--◂ épineⓋ 04:14, 22 May 2018 (UTC)
I've made some changes today which should make the picture appear more often. The problem is that there are so many ways to "hide" an image in an infobox. One example I've just added support for: {{Infobox ship image}} names its parameter Ship image= rather than the more common image=. I can't just take anything containing the string "image", because we don't want parameters such as map_image. If you're seeing the problem on a non-English wiki then it may be an internationalisation issue (because infoboxes on that wiki name their parameters using non-English words that I didn't think of). Certes (talk) 15:09, 22 May 2018 (UTC)
It looks like the code for the FA star is being transcluded on Portal:Underwater diving. I checked by taking out that specific transclusion (Underwater diving), but it took a few tries, so I think it does it for all FA articles - there can be up to three in that portal. Cheers, · · · Peter (Southwood)(talk): 11:58, 23 May 2018 (UTC)
@Pbsouthwood: Clicking the star takes me to WP:Featured pictures, so I think it may be coming from an image rather than a transcluded article excerpt.
Most FA display their star using {{featured article}} at the top (which gets removed along with the rest of the heading tag junk) or at the bottom (which gets removed along with the non-lead text). It is possible that someone buried the tag in the middle of the lead instead, in which case it will display the star, but without knowing which article is causing a problem that's hard to verify. I've changed the module to explicitly remove that sort of template wherever it appears, just in case. Certes (talk) 12:46, 23 May 2018 (UTC)
On a very few pages, custom templates are used that are not needed for Transclude lead excerpt (such as {{Mithun Chakraborty sidebar}} and although templates like this can be removed, we could not always prepare for all eventualities. I propose that a parameter is added that allows removal of certain templates that would be included. Dreamy Jazz (talk) 22:04, 24 May 2018 (UTC)
@Dreamy Jazz:Fixed. Portal:Bengali cinema/Selected biography/7 no longer transcludes the sidebar from Mithun Chakraborty. That article positions the sidebar unusually at the end of the lead. The module removes templates found before the lead, but retains templates within or at the end of the lead unless they're on a blacklist. I've now changed Module:Excerpt to blacklist sidebars. Certes (talk) 22:31, 24 May 2018 (UTC)
This template does not remove {{UN_Population|ref}}
@Dreamy Jazz: Done. I've updated the module to keep such templates but strip out a |ref or |shortref parameter. There's a risk of catching parameters where "ref" has some other meaning, but I think it's low. Several templates work this way, including some obscure ones, so I don't think it's practical to maintain a list. Certes (talk) 10:59, 25 May 2018 (UTC)
Yes, the module removes templates which occur before the lead, because they're usually not wanted. Here, {{nihongo}} is part of the lead but that's not obvious. Perhaps I can persuade it to retain templates which are before but on the same line as the first unadorned text in the lead. Certes (talk) 20:22, 29 May 2018 (UTC)
This also seems to cause a problem with the paragraph transclusion (see Portal:Ancient Japan/Selected article/5). It seems to include a small amount from the fourth paragraph, even though I have defined it not to. This problem only seems to happen if the paragraph parameter is defined (not seen at Portal:Ancient Japan/Selected article/1). Dreamy Jazz (talk) 20:25, 29 May 2018 (UTC)
That's a useful example and I think it shows the same problem. Like paragraph 1, paragraph 4 starts with a template. Templates at the start of a paragraph are not yet being recognised as part of the paragraph, so the module does not understand that paragraph 4 has already started. Certes (talk) 20:33, 29 May 2018 (UTC)
@Dreamy Jazz:Fixed. Templates which are followed by text which is not a template are now treated as part of that text rather than as a template. Certes (talk) 21:43, 29 May 2018 (UTC)
Removal of "citation needed"-like templates
@Certes: I know the module already removes {{cn}} but it misses several other in the same vein. See here for details. — AfroThundr (u · t · c) 11:46, 1 June 2018 (UTC)
I assume that the "paragraphs" parameter will select the same paragraphs for all articles in the group, and the "files" parameter will do the same. How is this useful? Most articles would need different paragraphs and files for optimum display. This feature only makes sense for a single selected article. If I am wrong, an explanation would be helpful. Cheers, · · · Peter (Southwood)(talk): 06:56, 30 April 2018 (UTC)
Paragraphs= and files= exist because the Lua module is shared with {{Transclude lead excerpt}} and it was easier to provide those features than to remove them. Paragraphs=1,3,5 makes no sense unless the articles being transcluded all follow one strict pattern. However, something like paragraphs=1-4 may be useful to keep extracts reasonably short without tailoring each one individually. (If the lead contains fewer than four paragraphs, it quietly includes them all.) Files=1 includes the article's first image if any. That sounds like a good thing to do, especially if we enhance the module to lift an image from the infobox.
As an aside, I've called the parameter files= rather than images= because it counts audio samples, etc. too. (They're hard to distinguish, especially if a new filename suffix appears in future.) I could add images as a synonyms but it might be confusing when it pulls out something that's not a picture.
At least this way anyone who thinks my way can use the parameters, and anyone who prefers your approach can omit them. Certes (talk) 08:54, 30 April 2018 (UTC)
Update: despite its name, files= now chooses only images, not other media such as audio files. Certes (talk) 16:52, 1 June 2018 (UTC)
I'm currently seeing that images grabbed from articles will give the full thumbnail in return. I would ask if it's possible to resize an image when calling the template but I believe unlogged users will still see the default thumbnail size? – TheGridExe (talk) 12:50, 1 June 2018 (UTC)
I wrote the image code behind this template but I don't know the answer to that question. Perhaps someone else can help. One problem is that the template chooses randomly between articles with different images, so it's hard to find one size that suits them all. Originally I left the images at the same size used in the article but that was unpopular, so I picked thumbnail as a good compromise. Certes (talk) 16:52, 1 June 2018 (UTC)
Are the tags {{what?}} and {{Primary source inline}} supposed to be removed?
Comment: It seems to be due to the way that this template is recognizing the wikitable. It seems to see each row of the wikitable is seen as a paragraph. Dreamy Jazz (talk) 20:19, 29 May 2018 (UTC)
I have reverted the changes on the selected article. Dreamy Jazz (talk) 20:20, 29 May 2018 (UTC)
I never thought of someone putting in a table before the lead had even started! I think it would normally work, but in this case it's complicated by the presence of images which are stripped out leaving a barely legal table syntax. Unless someone has a bright idea, I think we may have to draw the line at this one: the risk of adding extra complexity to handle such an unusual format would probably outweigh the advantages. Certes (talk) 20:29, 29 May 2018 (UTC)
These templates respect the noinclude tag, so one option is to mark <noinclude>problematic text</noinclude> thus. Certes (talk) 11:27, 2 June 2018 (UTC)
Hi Certes. The template appears to transclude the short description and DISPLAYTITLE templates. These are not useful. DISPLAYTITLE seems to be ignored and only triggers a warning in edit mode, but the transcluded short description overrides the portal's own short description, as it is later in the page code, which is a problem. Cheers, · · · Peter (Southwood)(talk): 14:47, 18 June 2018 (UTC)
@Pbsouthwood: Thanks; I'll add them to the removal list. I was hoping that they would only appear before the lead. To help with testing, please do you have any real-world examples of where they cause a problem? Certes (talk) 15:19, 18 June 2018 (UTC)
Certes, I noticed them while working on Portal:Underwater diving. One of the cases that came up was Kursk submarine disaster for DISPLAYTITLE (it has italics in the title). The short description problem is not happening at the moment. Either it has been fixed or it is intermittent and only happens in some cases. Unfortunately I didn't record the short descriptions that came up so can't track down which articles were responsible. I will do that if it happens again. · · · Peter (Southwood)(talk): 16:02, 18 June 2018 (UTC)
Aha! Kursk is (quite correctly) using the {{DISPLAYTITLE:… magic word rather than the {{DISPLAYTITLE|… template. I'll check for that too. Certes (talk) 16:09, 18 June 2018 (UTC)
I've set up excerpts from Kursk and the similar Sinking of the Rainbow Warrior in my sandbox and they work fine. My sandbox also wikilinks the Warrior article near the start of the lead, but the Portal page doesn't. Failing to wikilink italicised titles is a bug that I fixed on 22 May. I suspect that purging the portal page would fix this problem. I've not done that as it would destroy the evidence, but please feel free to go ahead. You may have to try a few times before an italic title appears! Certes (talk) 16:39, 18 June 2018 (UTC)
Warning: Display title "<i>Kursk</i> submarine disaster" was ignored since it is not equivalent to the page's actual title. is still coming up occasionally in edit mode. Purging resets the random selection, so it will only happen when the Kursk or Rainbow Warrior article is selected. The short description override could actually cause confusion in a mobile search, so relatively important to be fixed, but it has not come up recently, so may be fixed.
Warning: Display title "Portal:Underwater divingUnder '<b></b>" was ignored since it is not equivalent to the page's actual title. has also come up a couple of times, but may be unrelated. I have no idea what is causing it or how to track it down. I assume it is called from one of the randomising templates, because it only comes up occasionally, but I don't know of anything in any of the lists for random display that would refer to a portal. · · · Peter (Southwood)(talk): 04:44, 19 June 2018 (UTC)
@Evad37: From experiment, this seems to be caused by local expandedContent = frame:preprocess(text), which now runs on every template to check that it's not an infobox. Can we run this check in some other way? This is also an expensive call, which may be a problem for portals that quote from many articles each with many templates. Certes (talk) 10:02, 19 June 2018 (UTC)
Would using frame:expandTemplate be any better? It would just need the template name to be extracted, which isn't difficult (the args can probably just be left out, since it only needs to check if there's a table with the class 'infobox' in the output). - Evad37[talk] 10:44, 19 June 2018 (UTC)
Probably. It's worth a try if we can't find a cheaper way. DISPLAYTITLE is a special case because it's normally used as a magic word (with colon), though the deprecated template of that name (with |) also works, but it would be good to solve the more general problem. I think most infoboxes produce a table with class list including infobox (for example, mw.getCurrentFrame():expandTemplate{title = "Infobox journal"} produces <table class="infobox hproduct"…), so we could match something like <table class="[^"]*infobox in the output (%s* etc. omitted for clarity). Certes (talk) 11:01, 19 June 2018 (UTC)
Some boxes construct a table out of wikitext though, e.g. {{Subspeciesbox}} calls {{Taxobox/core}} which outputs {| class="infobox biota"... rather than a <table> element - Evad37[talk] 11:33, 19 June 2018 (UTC)
I think that's now all fixed. I've also blocked a few more reference syntaxes. Portal:Underwater diving still shows one ref, but that's from #section-h:Outline of underwater diving and not from this template. Certes (talk) 10:47, 20 June 2018 (UTC)
On a related point, Portal:Underwater diving reads {{Transclude random excerpt | Kursk submarine disaster#Salvage operation…. #Salvage operation is ignored. Respecting #section is an interesting idea for an enhancement but I'm not sure it would help here. The section begins The Russian government committed to raising the wreck… and it would be a while before it reminds the reader what the article is about. (|section= works but is not designed for random picks as it would seek the same section name in all articles.) Certes (talk) 10:02, 19 June 2018 (UTC)
I understand that as of now the template cannot source images from the infobox. However, the image in the infobox is often the most representative image on the topic. Given this, can we have an enhancement in this template to include the images from infobox subject to free image license verification? Arman(Talk) 05:13, 21 June 2018 (UTC)
The template does attempt to source images from the infobox, but fails on some specialised infobox templates which use unusual syntax. Please can you link to an example where it fails? We may be able to enhance the template to handle that case. Certes (talk) 10:36, 21 June 2018 (UTC)
Would it be possible to add a minimum article quality parameter? For example if you only want to display articles of B-class or better. eg | minimum-class=B . Alternatively a parameter set that specifies all the classes that may be displayed, eg | class=C,B,GA to display only articles of classes C, B, and GA would be more versatile, though I don't see any obvious use for it. Cheers, · · · Peter (Southwood)(talk): 05:03, 25 June 2018 (UTC)
This template is designed to work in cooperation with JL-Bot, which can limit the source list to good and/or featured articles. However, it may be possible for Module:Excerpt to limit the scope as you suggest. We'd have the usual problem that editors find inventive alternatives to adding the expected text string, but at least that would be a right-side failure in that we'd display some other good article instead. It may involve substantial work, both in coding and in parsing talk pages at run time, especially for an article such as The Hague which has been assessed by multiple wikiprojects. Certes (talk) 11:36, 25 June 2018 (UTC)
That should also work. Can JL-Bot be requested to include B-class? I don't remember that as an option, though it might be less processor intensive depending on how it identifies the class. If the check was made according to the category used by the WikiProject associated with the portal to filter out the unwanted articles it might be quicker. Actually I have no idea how JL-Bot goes about its allotted tasks, so may be making useless suggestions. Anyway, just in case something might work... Cheers, · · · Peter (Southwood)(talk): 17:53, 25 June 2018 (UTC)
Some article excerpts are failing to acquire any image, despite a non-map image being present eg Dubh Artach, Cuillin, Fingal's Cave
Thanks for any assistance with these! Espresso Addict (talk) 20:37, 12 July 2018 (UTC)
@Espresso Addict: I've updated the image detection code, can you check if there's any still missing an image? - Evad37[talk] 08:58, 13 July 2018 (UTC)
Thanks, @Evad37:, that looks a lot better. As far as I can tell (the script started timing out all the time before I'd finished), the only ones missing an image now are Whisky Galore! (1949 film), which I assume is because it's correctly assessing the poster as copyright, and all the ones where the only infobox image is a generic map with a location dot.
The only new problem I saw is that the caption for the Orkney image is wrong -- it shows the flag, but gives the caption for the coat of arms.
There are still the other two issues noted above, and in particular quite a few articles aren't linking besides those I noticed earlier -- I can do you a list if that would help? Cheers, Espresso Addict (talk) 14:01, 13 July 2018 (UTC)
For Orkeny, the problem is probably the order of paramneters – the module assumes that each caption parameter is for the immediately preceding image parameter. This edit [1] should fix it. I haven't yet looked at the code for the other issues, but the TOC is presumably from another template that needs to be removed by the code; and the article title link can probably be fixed for most cases by checking the non-disambiguate version of the title, i.e removing parentheses and everything after and including the final comma. - Evad37[talk] 02:46, 14 July 2018 (UTC)
Fixed the TOC issue - Evad37[talk] 03:16, 16 July 2018 (UTC)
@Espresso Addict: Fixed the bold text linking too (for most cases, probably) . - Evad37[talk] 03:50, 16 July 2018 (UTC)
Placement of image in excerpt
It looks like the images are placed where they are found, so if the firat image is below the lead in the original that is where it will display in the excerpt. I suggest that stacking them on the right starting at the top of the box is more generally useful. Having a moderately long excerpt with an image below it creates a lot of non-functional whitespace. Cheers, · · · Peter (Southwood)(talk): 08:18, 13 July 2018 (UTC)
An interesting idea. We'd be overriding layout choices made in the article, but maybe they don't apply to the excerpt. Perhaps we should stack images in the default way, respecting the module's fileargs=left flag or any other settings. (Arabic Wikipedia uses a copy of this module and prefers images on the left.) Certes (talk) 08:42, 16 July 2018 (UTC)
Heavy metal has exactly the effect we should avoid. It would be nice to be able to choose which side the stack displays, but the images should generally not take up a lot more vertical space than the text, or we end up with big whitespace. Obviously this depends on page width, but for a wide screen it would be preferable if there is not an extra image below the bottom of the text. Alternating between left and right looks fine on a wide screen, but really bad when the text sandwiched between images gets narrow. Adjusting the display of images to respond to screen width would be nice, but is it feasible? Limiting column width might also be feasible. It may be good for redability - the eye finds the start of the next line more easily if the number of characters in the line is smaller. Some of this may be beyond the scope of templates, as the screen characteristics would have to be available.
I don't think that overriding display choices made in the article is an issue. We already do that by not displaying infoboxes. The article may continue for hundreds of lines below the lead. Our excerpts may not even include the whole lead. Stacked images is the easy way to give an unsurprising effect and the highest probability of consistently acceptable display. We can't expect it to be infallible. · · · Peter (Southwood)(talk): 16:22, 16 July 2018 (UTC)
@Pbsouthwood: Done That sounds very reasonable. I've updated Module:Excerpt to output images first. The change shouldn't affect left or right positioning. Certes (talk) 17:13, 16 July 2018 (UTC)
This list of possible near-future changes was inspired by talk above and at WT:WikiProject Portals. Comments are welcome. Certes (talk) 14:17, 16 May 2018 (UTC)
Internationalisation
As the module is being used on non-English Wikipedias, it might benefit from splitting into two pages: one for core functionality; one for language dependent text. One option is to have an optional submodule which simply defines a Lua table to override the English language defaults in the code. For example, Arabic Wikipedia can have something like langtext = { more = "اقرأ المزيد...", file = {"ملف" ,"صورة", "[Ff]ile", "[Ii]mage"} }. Where the module currently uses "Read more...", it can instead use (langtext.more or "Read more..."). Where it searches for File:Foo or Image:Foo, it can instead search for any of the given namespace patterns. There are other strings to include. The submodule will be optional (if it's missing, you get English) and should only need to be edited when a new string appears. This looks like the largest enhancement and will take quite a bit of work, so let's make sure the design is right first! Certes (talk) 14:17, 16 May 2018 (UTC)
Longer text
When transcluding an excerpt from an article with a short lead, you get a short excerpt. The preferred way to fix this is to edit the article to lengthen the lead, perhaps by moving text from a later section, as this will normally improve the article itself. No enhancements are planned in this area but this may change. Certes (talk) 14:17, 16 May 2018 (UTC)
I agree that this is the correct strategy.· · · Peter (Southwood)(talk): 13:42, 17 July 2018 (UTC)
Excluding files other than images
files= can transclude files other than images, such as audio files. There seems to be consensus for ignoring non-image files, so files=1 will transclude the first image, even if preceded by a non-image file. This is potentially a breaking change, e.g. someone may have coded files=2 to skip a leading audio file. Certes (talk) 14:17, 16 May 2018 (UTC)
Done: only files with image extensions now appear. Certes (talk) 15:43, 21 May 2018 (UTC)
Later images
There is some demand for transcluding images which appear in later sections of the article rather than the lead. This could be achieved without breakage by a new optional argument such as sectionfiles=true. It may be resource hungry but should not impact existing usage without the new flag. Certes (talk) 14:17, 16 May 2018 (UTC)
File:File:Foo.jpgAll look like good ideas. There are sometimes problems with images in infoboxes. Getting them to display would be great, but there is also the problem of occasional blank image display with File:File:Foo.jpg in the thumbnail frame (as shown at right). Cheers, · · · Peter (Southwood)(talk): 14:37, 16 May 2018 (UTC)
@Pbsouthwood: File:File:Foo.jpg is a separate bug, which I just fixed. If a portal still has that problem after purging, please give me a link. Certes (talk) 16:31, 16 May 2018 (UTC)
Non-random selection
Allow {{Transclude random excerpt}} or a new template to pick from a list in a non-random way, e.g. based on the week number or day of the year, to provide a slowly and predictably rotating carousel of extracts. Certes (talk) 22:41, 18 May 2018 (UTC)
Most articles have a beginning like A pig is..., with the article title or a synonym in bold. We could wikilink this to the article itself. It's not quite as easy as it looks; see for example Elton John. Certes (talk) 18:06, 20 May 2018 (UTC)
In Portal:Royal Navy the lead excerpt is displaying File:Flag of Scotland (traditional).svg which is not visible in the lead, infobox or any of the following sections. The only place it exists is in footnote 5, which is linked from the infobox under "insignia", so I assume it is getting it from there. It is not even the first image in the infobox, so I can't guess why it is getting it and not one of the others. Maybe some weird infobox coding. Cheers, · · · Peter (Southwood)(talk): 14:33, 17 July 2018 (UTC)
@Pbsouthwood: There were actually two problems: the first image wasn't being detected, and the order they were being transcluded in was getting mucked up. Both issues have been fixed now – File:Naval Ensign of the United Kingdom.svg is now detected by the module, and the order is back to being the same as the order they appear in the wikitext. - Evad37[talk] 07:54, 19 July 2018 (UTC)
Bug with {{Transclude lead excerpt}}: Lua error: bad argument #2 to 'find' (pattern is longer than 10000 bytes).
Lua error: bad argument #2 to 'find' (pattern is longer than 10000 bytes).
Trace:
1. [C]: in function "v"
2. mw.ustring.lua:84: in function "find"
3. Module:Excerpt:97: in function "argimage"
4. Module:Excerpt:314: in function "parse"
5. Module:Excerpt:438: in function "main"
6. Module:Excerpt:515: ?
7. (tail call): ?
8. mw.lua:511: ?
9. [C]: ?
@Evad37, Certes, and Dreamy Jazz: I changed it using a very ugly hack. basically, we need a way to cut the size of the input before the string processing starts, or, a way to pass only the lead section to the module using LST. Frietjes (talk) 22:29, 18 July 2018 (UTC)
I've put a suggested fix in Module:Excerpt/sandbox. I haven't tested it properly as that would require reverting the portal change. I won't be online much for the next week but please copy to Module:Excerpt if someone else can look after it. Certes (talk) 22:54, 18 July 2018 (UTC)
thank you. FYI for The Transhumanist and others, you can't use {{PAGENAME}} when the page includes an apostrophe or ampersand. this could be addressed in the module (e.g., by changing the & to & within the module), or just conclude that it's "garbage in" = "garbage out" and address it at the module call level as I have done here and here. Frietjes (talk) 13:45, 19 July 2018 (UTC)
Hey! {{Transclude lead excerpt}} seems to not ignore {{Efn}} (and possibly its sub-templates as well) in the ckb wiki (check here) I don’t know if this problem is present in the enwiki though.--▸ épinetalk♬ 13:11, 6 August 2018 (UTC)
Efn is first in the list of unwanted templates in Module:Excerpt on enwiki. I added it on 25 April. Perhaps ckb is using an older version. Certes (talk) 13:22, 6 August 2018 (UTC)
The template {{Unreliable source?}} is used to mark a reference as potentially unreliable. This shouldn't be included when the reference it is referring to is not included.
@Evad37: The {{efn-lr}} is nested within {{cite-web}}. I'm busy today but will look at it tonight unless you manage to fix it first. Certes (talk) 07:56, 2 September 2018 (UTC)
efn-lr was actually buried in {{nihongo}}, which we're correctly leaving in. I've changed the code to recursively remove unwanted templates which are nested within wanted ones. I've also added templates beginning with "cite-" (hyphenated) to the blacklist. The one visible side-effect is that fewer images appear when the testcases page transcludes from Royal Navy: the excerpt no longer displays historical flags from the footnote. Certes (talk) 21:00, 2 September 2018 (UTC)
{{Transclude list item excerpt | 1=Outline of {{PAGENAME}} | section= Environment of {{PAGENAME}} | paragraphs=1-2 | files=1 | fileargs= | more= | errors= }}
which reduces to
{{Transclude list item excerpt | 1=Outline of Milan | section= Environment of Milan | paragraphs=1-2 | files=1 | fileargs= | more= | errors= }}
which causes a Bogus file options lint error, viz: "A typical foggy day in central Milan." The bogus file option text comes from Milan#Climate, which begins,
[[File:Milan foggy panorama with Velasca Tower.jpg|thumbnail|A typical foggy day in central Milan.]]
So, Template:Transclude list item excerpt is sucking up valid Caption parameters from articles and stuffing them into file markup, but followed by a pipe, or in some way not in the final position, so the Caption parameter becomes bogus. —Anomalocaris (talk) 18:50, 27 August 2018 (UTC)
The presence of an empty | fileargs= causes Module:Excerpt to add a pipe at the end of the image code: [[File:Milan foggy panorama with Velasca Tower.jpg|thumbnail|A typical foggy day in central Milan.|]]
. Either the module should be modified or the calls should omit the parameter when it's empty. Modifying the module is best but I'm not a Lua coder. PrimeHunter (talk) 19:48, 27 August 2018 (UTC)
Yeah, this a Module:Excerpt issue. Though Module:Excerpt slideshow does already trim whitespace and remove empty parameters (via the function cleanupArgs) before passing any through to Module:Excerpt, so what PrimeHunter describes might not be the only issue - Evad37[talk] 01:10, 28 August 2018 (UTC)
I don't do Lua, but I have found similar issues in templates. One way to fix it is to test for the presence of a variable, and then insert the pipe for that variable's value only if the variable is present. Otherwise, omit the pipe. Here's an example fix. Maybe Trappist the monk could help us with the Lua. – Jonesey95 (talk) 06:31, 28 August 2018 (UTC)
Fixed (at least the empty fileargs issue) with [2]. Lua is a bit funny in that empty strings are truthy instead of falsey, and that (unlike regular temlplate coding) parameters don't have whitespace trimmed or present-but-empty parameters ignored. You have to do it yourself, either at the start with something like the cleanupArgs function I use in modules I create, or else when needed (which isn't always obvious, like in this case) - Evad37[talk] 07:08, 28 August 2018 (UTC)
Thank you, Evad37. That appears to have worked. The number of Portal pages in the "Bogus file options" "category" (still not an actual category, but we can hope for it someday) has been greatly reduced. The ones that are left appear to be actual errors, as far as I can tell (although Portal pages are ... strange). – Jonesey95 (talk) 19:56, 28 August 2018 (UTC)
The first [[Brookville Liberty Modern Streetcar|Brookville Liberty]] streetcar for Oklahoma City, a few weeks after its arrival. It wears one of three different paint schemes planned.
| caption = The first [[Brookville Liberty Modern Streetcar|Brookville Liberty]] streetcar for Oklahoma City, a few weeks after its arrival. It wears one of three different paint schemes planned.<ref name="deliveries"/>
...}}
There are 14 affected items in the Portal namespace. —Anomalocaris (talk) 23:50, 2 September 2018 (UTC)
Entirely unrelated to flex columns, which just wraps whatever content it is given in divs with flexbox styling via TemplateStyles. Putting {{Transclude lead excerpt|Oklahoma City Streetcar|files=1}} into Special:ExpandTemplates shows that the file syntax this modules is generating from the infobox parameters is
[[File:OKC Streetcar (38923640550).jpg|The first [[Brookville Liberty Modern Streetcar|Brookville Liberty]] streetcar for Oklahoma City, a few weeks after its arrival. It wears one of three different paint schemes planned.|alt=on|allk=on|first=imp}}|thumb]]
which makes me think that it's somehow mistaking on|allk=on|first=imp}} (from the |gauge= parameter further down the infobox) for the alt text, which has in fact been left blank. - Evad37[talk] 01:43, 3 September 2018 (UTC)
Should be fixed with this edit - Evad37[talk] 08:33, 3 September 2018 (UTC)
The template {{self-published source}} is not removed when transcluding an article lead. Seeing as the template being there refers to a reference not seen in the lead, it could confuse readers and so shouldn't be there. Thanks for continual improvements. Dreamy Jazz 🎷 talk to me | my contributions 09:09, 11 September 2018 (UTC)
@Dreamy Jazz: I've removed the tag. Thanks for the report. I'm guessing that this affected Portal:United States Territories but I can't see any improvement: perhaps it messed up a randomly selected article which isn't appearing today. Certes (talk) 10:51, 13 September 2018 (UTC)
@Certes: The article in question is Kingman Reef. However, it does not seem to be removed. Have purged the page etc., but it still remains in the transclusion. Dreamy Jazz 🎷 talk to me | my contributions 10:59, 13 September 2018 (UTC)
@Certes: Both. First there's the stuff this module does, from Module:Excerpt slideshow calling Module:Excerpt's main function to get the excerpt. Later, Module:Excerpt slideshow does further removal of block-level templates and tables, which could otherwise mess up the gallery syntax. In this case the problem is definitely with Module:Excerpt, since it can be replicated with {{Transclude lead excerpt | paragraphs=1-3 | files=1 | more= | Kingman Reef }} - Evad37[talk] 12:55, 13 September 2018 (UTC)
Fixed. I forgot to escape the hyphen (should be self%- not self-). Certes (talk) 13:04, 13 September 2018 (UTC)
Vogelsberg
@Evad37: As you've spotted, this template breaks the Vogelsberg media link. The bug is in the bit that turns the bold title into a wikilink (search for "replace the bold title"). Commenting out that block of code removes the problem (but of course breaks other articles). I'm busy fixing a different bug at the moment but we should think about how we can detect cases like this and refrain from making an unwanted link. Certes (talk) 11:06, 13 September 2018 (UTC)
What should appear in the extract for Vogelsberg? The article itself is unusual in that the bold title in the lead isn't unlinked text but a wikilink to a useful destination. Should the extract look the same, with no link to the article? Should the link lead to the article, with no way of playing the sound? Perhaps the sound symbol should link to the sound, and the text to the article, but that's not how {{audio}} normally behaves and I'm reluctant to code for one very special case. Certes (talk) 13:41, 13 September 2018 (UTC)
@Certes: Hello. Would you mind moving the fix over from the sandbox that fixes the issues with Morpeth, Northumberland. I noticed that for the sandbox module the image works as expected, but not for the "live" version. It does cause big problems with the look of the portal when the town's article is shown in the portal. See my sandbox for the comparison. Thanks, Dreamy Jazz 🎷 talk to me | my contributions 17:56, 6 December 2018 (UTC)
@Dreamy Jazz: Done. Most of the work is by Evad37. I'm not sure how far Evad got with testing but it looks good to me. Certes (talk) 18:14, 6 December 2018 (UTC)
@Kusma: The module attempts to parse wikitext. Sadly we don't have the resources to write a bulletproof wikitext parser in Lua, let alone run it many times in the 10 seconds allowed. I was surprised to find this module deployed on 6,600 pages displaying (I guess) close to 100,000 articles, and a few of them use syntax we didn't think of. WikiProject Portals terms this particular error a "splurge" and you'll find plenty of discussion about it in the project pages. Usually we have enhanced the module to handle whatever oddity that article contains; occasionally we had to edit the article to avoid the awkward construct. Most errors are easily fixable and I used to be quite diligent about doing this promptly, but I am ashamed to say that the constant barrage of ritual humiliation such as PortalShit3.png has removed my motivation to bother. Certes (talk) 11:04, 3 April 2019 (UTC)
You are right to say that the error is random. When the module finds more articles than it has been asked to display, it chooses randomly, and the misbehaving article only sometimes comes up. Certes (talk) 11:06, 3 April 2019 (UTC)
@Certes: I apologise if it looked as if I was demanding anything, I just tried to make a bug report in case this is an issue that can be fixed in general. The file name used is certainly not optimal. In the portal at hand, I think part of the problem comes from using New England Revolution in the slideshow (in addition to using it as the introduction box). But I could be wrong, and playing around for five minutes failed to produce a solution, unfortunately. Either it is this duplication that causes problems or something else in the article isn't escaped properly? (sorry that it is still not a very good bug report). —Kusma (t·c) 14:45, 3 April 2019 (UTC)
That's a good bug report, and thank you for raising it. I hope that a more dedicated editor will step in and fix it. Once the deletion squad has had its fill of portals and moves on, I'll come back and look at whatever is left, but for now I'm concentrating on improving pages that are not about to disappear. Certes (talk) 14:51, 3 April 2019 (UTC)
Limit word count
Is there any way to limit the word count of the excerpt? Hecato (talk) 23:28, 21 June 2019 (UTC)
@Hecato:|paragraphs=1-n will limit the number of paragraphs. There's no explicit word count limit, as that might produce an excerpt which ends abruptly in the middle of Certes (talk) 10:07, 22 June 2019 (UTC)
I don't think the ending in the middle of a sentence thing would be too bad if there was a situation like: "word word word... (Read More)". Maybe I will hack something together myself. Thanks for the quick answer! Hecato (talk) 10:21, 22 June 2019 (UTC)
Grant to improve and spread excerpts
Hi! I just requested a grant to improve and spread excerpts (not just this module but the more general concept of excerpts, see Wikipedia:Excerpts) in the English Wikipedia and five other wikis. If anyone is interested and would like to leave a comment in the grant's page, it would be most welcome. Thanks! Sophivorus (talk) 21:56, 13 December 2019 (UTC)
Section excerpts
The Template:Excerpt allows users to transclude a lead section as well as any other section. For lead sections, the template uses this module, but for other sections, it uses the #lsth parser function. This has two problems: first, that #lsth doesn't have any of the goodies of this module (such as removing unwanted elements) and second, that #lsth transcludes the desired section but also all subsections, which often conflict with the heading hierarchy on the transcluding article. This can be avoided by specifying the title of the first subsection as a stop mark, but this makes it more complicated to use and unreliable. In short, it would be really useful if this module could be extended to allow transcluding specific sections. I plan to do so myself eventually, but in the meantime I wanted to mention it. Cheers! Sophivorus (talk) 18:27, 7 January 2020 (UTC)
@Sophivorus: The module already supports section transclusion by giving the page name in the form Article#Section. It does include subsections of the section, but it shouldn't be too hard to add another option to exclude them. Certes (talk) 18:40, 7 January 2020 (UTC)
@Certes: Awesome, thanks for the heads up!!! I did some tests and it seems that the module does exclude subsections (yay!). Sophivorus (talk) 23:10, 7 January 2020 (UTC)
Option to keep references
Hi! This template is starting to get used in the main namespace through the Template:Excerpt. However, the current version of the module strips references, which are are important in the main namespace. Maybe this is an easy option to add for someone familiar with the module? Else I'll dive into it myself in a few days, thanks!!! Sophivorus (talk) 15:27, 13 November 2019 (UTC)
@Sophivorus: This module was designed for the specific needs of portals, and labeled section transclusion (see Help:Magic_words#Other) might meet the needs of the main namespace better. If you do need to edit the module, function striptemplate is the place to look. Depending on the exact output you want, it probably needs to look for a new "keep refs" flag and, if found, limit table unwantedTemplates to a subset of its current values and skip much of function striptemplate. Certes (talk) 16:14, 13 November 2019 (UTC)
@Certes: LST has the problem of transcluding unwanted elements such as hatnotes, infoboxes, multiple images, etc. so I think modifying this module is the way to go. Thanks for the guidance! Sophivorus (talk) 16:48, 13 November 2019 (UTC)
@Certes: Done! I did my best to ensure the changes cause no bugs elsewhere, but I would be very grateful if you could take a look too. Thanks! Sophivorus (talk) 18:15, 7 January 2020 (UTC)
@Sophivorus: I just had a quick look at the changes. That all looks good and shouldn't break anything for existing users. My one concern is that by skipping striptemplate completely you retain all templates listed in unwantedTemplates, including ones such as {{Citation needed}} which you may not want. A few of them have subtle side-effects, notably {{Good article}} which would classify the page you transclude onto as a GA, and DEFAULTSORT which would change the sort code of the page you transclude onto. striptemplate itself also removes a few non-reference templates such as audio files which you may not want but would retain if you skip the function entirely. But perhaps that's exactly what you need; I don't understand your requirements fully. Certes (talk) 01:02, 8 January 2020 (UTC)
Hi! I just requested a grant to improve and spread excerpts in the English Wikipedia and also five other wikis. If anyone is interested and would like to leave a comment in the grant's page, it would be most welcome. Thanks! Sophivorus (talk) 21:55, 13 December 2019 (UTC)
Tables removed
The module is stripping tables completely. For Template:Excerpt this is not desirable, because often what you want to transclude is a table, so I'd like to add an option for keeping tables. However, I browsed the code of the module for quite a while and I can't seem to find the relevant lines. Do you know which ones are they? Thanks! Sophivorus (talk) 23:10, 7 January 2020 (UTC)
@Sophivorus: To be honest I'm not sure why tables are removed. It's not deliberate but seems to be a useful side-effect for portal purposes if not for you. It may be because a table tends to contain several paragraphs and the excerpt reaches its paragraph limit mid-table. There is code in there to strip out incomplete structures so we don't end up with half a template (or half a table) dragging in the text after the excerpt. Certes (talk) 01:25, 8 January 2020 (UTC)
@Certes: After several tests I managed to figure out the relevant part of the script, but not the exact line. It seems it's the "repeat" loop in the "parse" method. The %b{} there is matching tables and removing them. If I set local token = false in the first line of the loop, then the tables are preserved. Will continue later, cheers! Sophivorus (talk) 23:01, 9 January 2020 (UTC)
@Sophivorus: Can you tell me what article you're trying to make an excerpt from? Tables do appear, see example, but perhaps your case is different in some way such as having a long table which gets truncated and removed as incomplete. Certes (talk) 23:38, 9 January 2020 (UTC)
@Certes: Interesting! It seems that tables with headings (cells starting with !) are not transcluded but tables without are transcluded fine. I modified your sandbox example to illustrate. Cheers! Sophivorus (talk) 00:26, 10 January 2020 (UTC)
@Sophivorus: Not quite: I changed the headers in sandbox2 to ordinary rows and the table appeared. The important difference is that my example has blank lines above and below the table but yours doesn't. Paragraphs which consist entirely of templates are removed – they're usually unwanted hatnotes etc. – and the same rule is being applied to tables. If a paragraph consists only of a table then it gets removed. If the table is part of a paragraph, by having text immediately above or below it with no separating blank lines, it is checked against the list of unwanted templates (refs etc.) and kept because it isn't unwanted. In your South America example, the entire excerpt is one paragraph consisting only of the template {{Historical populations}}, which gets removed. The module does not even get as far as discovering that the template would create a table. Certes (talk) 01:13, 10 January 2020 (UTC)
@Certes: I understand, thanks for figuring that out. Then it seems that this will require two separate fixes, one for regular tables and another for stand-alone templates that we want to keep. The first one should be easier so I'll start with that. Sophivorus (talk) 11:10, 10 January 2020 (UTC)
Existing uses (almost entirely portals, 90%+ ledes rather than sections) probably don't want to see tables. The excerpt may even be transcluded within a table of excerpts. If you change the handling then tables should probably be treated like references: removed by default but retained if a new flag is given. By the way, I generally develop in Module:Excerpt/sandbox and view Template:Transclude lead excerpt/testcases etc. with the sandbox to make sure I've not broken anything. Certes (talk) 11:24, 10 January 2020 (UTC)
@Certes: Thanks for the sandbox tip! I added a testcase for tables at Template:Transclude lead excerpt/testcases2#Tables and did some changes to Module:Excerpt/sandbox so that it keeps plain tables (diff). You'll notice that I haven't added a keepTables flag like you suggested. I have no problem adding it, but first I wanted to ask you if you really think it's necessary, because 99% of leads don't contain tables, so keeping tables by default won't affect leads, and as to sections, I can imagine that if a portal ever wants to feature a section containing a table (or even a section consisting of only a table) then keeping the table would be desired behavior. In other words, I can't imagine how keeping tables by default could be a problem. But if you think otherwise, I have no problem adding the flag. Also, one more thing: I checked my changes against your testcases, and I noticed that Template:Transclude lead excerpt/testcases#SS Vienna (1873) is trouble (if you check the wikitext at SS Vienna (1873) you'll immediately understand why). I think the solution will be to complicate my simple code to account for this kind of situation, but I wanted to check with you before doing so. Thanks for all your help, cheers! Sophivorus (talk) 17:25, 21 January 2020 (UTC)
@Sophivorus: Good spot with Vienna. The change you're making will keep tables intact, including references, infoboxes, etc. Portals don't want the references and (I think) no one wants the infoboxes. As you say, doing something cleverer may be complicated. Certes (talk) 17:50, 21 January 2020 (UTC)
@Certes: I improved my little snippet (diff) so that now references are removed. The only remaining problem would be that infoboxes inside templates (such as the one at SS Vienna (1873)) are kept, correct? I agree that removing such infoboxes would be complicated (at least for my level of knowledge of the module and Lua) so I would be happy to just add the keepTables flag for use at Template:Excerpt, and keep the current behavior of removing all tables on portals. Ok? Sophivorus (talk) 19:22, 21 January 2020 (UTC)
@Sophivorus: I think that would be safest. With hindsight, I should have thought about tables earlier and handled them better, but now maintaining backwards compatibility is important. Finding the end of the table should just be a matter of checking that {| comes next, as you already do, and parsing %b{} from that point. Searching for |} would be more problematic because of null parameters like {{Foo|}} and nested tables. Certes (talk) 20:38, 21 January 2020 (UTC)
@Certes: I added the keepTables check to Module:Excerpt/sandbox. I also renamed the unwantedTemplates array to unwantedInlineTemplates, moved it to the top, defined a wantedBlockTemplates array and a fileNamespaces array. The idea behind these arrays is to start separating what must be localized on a per-wiki basis from the rest, so that other wikis (such as the Spanish wiki where I'm most active) can use the module without modifying the source code, making it easier to adopt and to sync with the English version. The wantedBlockTemplates introduces new functionality: templates in their own line ("block" templates, as opposed to inline templates) are normally removed, unless they are listed there. This allows us to keep templates such as Template:Historical populations when transcluding stuff like List of continents by population#South America. I've done my best to ensure the testcases all look ok, and also added a few to Template:Excerpt/testcases, but let me know if you notice any issues or have any further concerns! As always, thanks for your help, know-how and wisdom, cheers!
@Sophivorus: I've not examined every detail but that looks good to me. It seems to achieve what you're trying to do without breaking anything. Elsewhere, the ArbCom case is reaching its conclusion and editors are resuming work on portals which use this module. However, this should mean deploying the existing templates on more pages, rather than enhancing the module to add new features. Certes (talk) 22:32, 27 January 2020 (UTC)
@Certes: Cool, then unless you object, tomorrow I'll update the module. Good to know about the ArbCom case, excerpts are the future! Sophivorus (talk) 22:59, 27 January 2020 (UTC)
"The time allocated for running scripts has expired."
See Portal:Speculative fiction/Selected works for an example. The random selection seems to work fine on the main portal page still. It just doesn't provide a list all of the articles as it used to. While the templates using this don't give a specific limit on number of entries, is there a limit? The page I linked has 226 entries. Please ping me on any replies to make sure I see them. Thanks! ···日本穣 · 投稿 · Talk to Nihonjoe · Join WP Japan! 17:35, 9 March 2020 (UTC)
@Nihonjoe: The only limit is the number of excerpts which can be processed in 10 seconds before the page times out. A selection of 20 to 50 is normal. I don't think I've seen more than 100 before. As you say, the actual portal only processes one of the 226 entries, so it shouldn't time out. Certes (talk) 18:24, 9 March 2020 (UTC)
@Certes: Thanks for the explanation. In the past, it's been able to display all of the entries, so that's why I asked. I guess it processes excerpts faster on some days. ···日本穣 · 投稿 · Talk to Nihonjoe · Join WP Japan! 20:35, 9 March 2020 (UTC)
i18n
Hi! So this module is used increasingly in the Spanish Wikipedia and I anticipate it will be used more and more in other wikis (whether I get the grant or not). So I think that it would be useful, as well as general good practice, to separate the module logic from the content that varies per wiki, such as the wanted templates, but also the file namespace name, and stuff like the name of the caption parameter that the module uses to build the image of the excerpt. This would allow the various wikis to easily keep up-to-date with the latest version of the module. I think the best way would be to have a JSON subpage with the localized data, and have the module pull the data from there. If there's no objection, I'll start working towards that goal eventually. Any help would be welcome, thanks! Sophivorus (talk) 16:57, 25 March 2020 (UTC)
@Sophivorus: The regex which matches the caption returns two captures: a position with () and a string. matchany() only handled a single capture, so "(foo*)" worked but not "(foo*)(bar*)". In the sandbox, I've enhanced matchany() to return all captures rather than just the first. Please check my change carefully before releasing, as I've not used that Lua idiom before. Certes (talk) 12:14, 4 April 2020 (UTC)
@Certes: Thanks a lot, that was sharp! I looked carefully at all the test cases and I notice no differences, so I think we're good to go. I enabled it on the Spanish Wikipedia and will enable it here soon if no issues occur. However, while reviewing the test cases, I noticed that some excerpts were not picking up captions, even with the current stable version of the module, for example Template:Transclude lead excerpt/testcases#Goa, Template:Transclude lead excerpt/testcases#Cotton Mather and Template:Transclude lead excerpt/testcases#Morpeth, Northumberland. Looking at the infoboxes wikitext, I see no obvious explanation, nor did I figure out the reason by analyzing the caption regexes and trying a few changes. Sophivorus (talk) 13:48, 4 April 2020 (UTC)
@Sophivorus: Thanks for checking these cases. Cotton Mather showed up a bug in parsecaption(), which also affected Geneva and Italian Renaissance. I've fixed it in the sandbox and those captions now appear. I think Goa pushes the template's limits. The first three images (beach, river, church) have no caption, or rather are covered by a single caption attached to the fourth image. The fifth image (grey map) is captioned with |mapcaption= rather than |caption=. We don't look for |mapcaption= and, if we did, we might have difficult associating it with the matching image. Morpeth, Northumberland combines both of those problems: it uses |static_image_caption=, and it covers multiple images. Certes (talk) 15:03, 4 April 2020 (UTC)
@Certes: Awesome fix, thanks again! Still, the [^=|]* part of the regex means "match anything that isn't an equals sign or a pipe", right? So shouldn't it match map in |mapcaption= and static_image_ in |static_image_caption=? If so, then the problem may be coming from elsewhere. One difficulty I'm having is that I don't know any way to debug. Suppose I want to dump the caption at any point in the execution to see whether it matched something or not. I tried the err() function and also debug.traceback() but when I went to the testcases I noticed nothing. Do you know how to do it? If you can help me out on this one, maybe I'll become more independent and productive in the future! Sophivorus (talk) 17:20, 4 April 2020 (UTC)
@Sophivorus: I added the following line to argimage() after captions[i] = parsecaption(caption) and previewed with this sandbox.
if mw.ustring.match(text, "static_image_caption") then error("\n*text=«"..mw.text.nowiki(text).."»\n*position=«"..tostring(position).."»\n*caption=«"..caption.."»\n*captions["..tostring(i).."]=«"..captions[i].."»\n*images[i]=«"..images[i].."»") end --TEMPORARY DEBUGGING
The "error message" showed that the caption was parsed correctly from |static_image_caption= and allocated to image[136] (136 is the character position within the wikitext). Next, I removed that debug and added after table.sort(keys):
local x=""; for key, val in pairs(images) do x=x.."\n*"..tostring(key).."⇒«"..val.."»"; end; error(x)
That showed me that there are two identical images found at positions 118 and 136 within the infobox, and the caption is assigned to 136 but it is 118 which is being displayed. The same image is detected twice because it is introduced by both |static_image_name= and <imagemap>. Perhaps this is just a pathological case; perhaps we need to do something more clever to detect overlapping pieces of wikitext which detect the same image in different ways. That might take a lot of code. Certes (talk) 18:16, 4 April 2020 (UTC)
Section transclusion
@Sophivorus: I have made a change to the sandbox which should cause section transclusion to include subsections. You can see the effect in Template:Transclude lead excerpt/testcases2#Cherry. I do not know whether anyone is relying on the behaviour of excluding subsections, so releasing the change might result in a request for a "section without subsections" option. Certes (talk) 21:26, 5 April 2020 (UTC)
@Certes: Awesome! Thanks for the lightning-fast response! An option to transclude sections without subsections will most definitely be necessary, because that's actually the main usage at the Spanish Wikipedia. I'll take care if you don't first. Sophivorus (talk) 21:52, 5 April 2020 (UTC)
@Sophivorus: In that case, we should probably continue to exclude subsections by default. I'll leave you to add something that works well for eswiki: perhaps replacing my new leadOnly parameter by leadOnly or not args["subsections"]. Certes (talk) 22:01, 5 April 2020 (UTC)
@Certes: Done! I changed the approach slightly and replaced the leadOnly parameter for a keepSubsections parameter similar to keepRefs and keepTables. I also defined a falsy() function to test for falsy values, since all the checks for empty strings, double negatives and default values in the Template:Excerpt were becoming a pain. All the testcases look good to me. Tomorrow I'll deploy on the Spanish wiki, cheers! Sophivorus (talk) 23:39, 5 April 2020 (UTC)
I got rid of the dependency of Module:Redirect. That dependency was a problem when porting to the Spanish Wikipedia and I anticipate it will be a problem for other wikis too. The dependency was very superficial and I was able to simplify the code by replacing the custom made function getRedirectTarget(title) with a call to title.redirectTarget which is not tagged as an expensive function. I also created Module:Excerpt/testcases, which are looking good. Cheers! Sophivorus (talk) 21:52, 5 April 2020 (UTC)
I moved all localization data to Module:Excerpt/l10n and modified the sandbox so that it requires it (diff). All testcases look good and I already deployed everything on the Spanish Wikipedia, so far without trouble. With this change, the module code of the Spanish Wikipedia and the English Wikipedia are identical (they just differ on the l10n subpage, obviously) so it becomes easy to keep them synced and cross-wiki development becomes less of a pain (yay!). There's still some more internationalization to do, but not completely essential, so we can do it gradually. Sophivorus (talk) 14:08, 6 April 2020 (UTC)
I'd like to move Module:Excerpt/l10n to Module:Excerpt/i18n, because it seems the standard naming convention on all other modules. If there's no objection, I'll do so with the next deploy. Sophivorus (talk) 01:35, 22 April 2020 (UTC)
W3C say that internationalization enables localization, i.e. making the enwiki module reusable is i18n and reusing it on other wikis is l10n, so the move works from an enwiki POV. Certes (talk) 06:20, 22 April 2020 (UTC)
The recent switch to Lua (implemented by Ahecht seems to have broken the image excerpting capability. See 2019–20_coronavirus_pandemic#Environment_and_climate, which previously transcluded the satellite photo at the top of that article. {{u|Sdkb}}talk 04:43, 24 April 2020 (UTC)
Ditto for 2019–20_coronavirus_pandemic#Education, only it looks like someone manually re-inserted it (which isn't great, since that'll have to be taken out again). {{u|Sdkb}}talk 04:53, 24 April 2020 (UTC)
@Ahecht: Got it; thanks! Were there any other sections where this was an issue? Overall, the switch may be good if it helps with PEIS, but I'm somewhat concerned that includeonly tags are now required at the source article or otherwise things will break. Editors are lazy and botch things up/ignore WP:FENCE, so a lot of monitoring is going to be required for maintenance. @Sophivorus: do you have any thoughts on this? {{u|Sdkb}}talk 19:21, 24 April 2020 (UTC)
@Certes, Sdkb, and Ahecht: I'm really happy that Ahecht has joined the development, welcome! In my opinion, the latest changes make it clearer than ever that this module needs to be restructured a little to allow for the fact that it's used on other wikis besides the English Wikipedia (mainly the Spanish Wikipedia, where it's used on almost a thousand articles and growing, but the Arabic Wikipedia seems to be following). The template-specific entry points, in particular, are only really relevant in the English Wikipedia, so I'm thinking about making a clear cut separation between those shorthand functions, and general, abstract, cross-wiki functions that may be used by any wiki to build their own templates. Sophivorus (talk) 20:00, 28 April 2020 (UTC)
Yes, the early code that I wrote is starting to suffer from the classic software problem that it does the job it was designed for but is less good at the task it has actually been used for. Certes (talk) 21:37, 28 April 2020 (UTC)
I'm glad you agree. Today I did some style changes to the sandbox while trying to get to know the code better (I hope you agree with the camelCase decision, but else please revert asap and I won't mind). However, I haven't been able to do the separation of concerns I was aiming for (yet). It's more delicate than it looks due to the widespread and varied use of this module already. I'll give it another go asap, cheers! Sophivorus (talk) 00:00, 29 April 2020 (UTC)
@Certes: Hi! After some thinking and experimenting, I came up with the following solution: I moved the enwiki stuff to Module:Excerpt/templates and added a few entry points to the module (sandbox) so that the submodule can use all the functionality it needs from the module. The templates would then call the submodule, rather than the main module. I tried it on the sandboxes of all the templates and the testcases are looking good. The advantage of this strategy, I think, is that enwiki may now develop it's template-specific function without any worries of i18n, while we can also develop the general, abstract, cross-wiki module with much more freedom. What do you think? Sophivorus (talk) 18:37, 10 May 2020 (UTC)
It's an interesting idea. The "Transclude...excerpt" templates are all trivial wrappers which just pass their arguments through to the module in the default way and are unlikely to change much. {{Excerpt}} now seems to have adopted that approach too. I haven't examined the modules which require("Module:Excerpt") in any detail, nor whatever uses non-English versions of the module on other wikis. Certes (talk) 19:00, 10 May 2020 (UTC)
Sophivorus I just tried out excerpting, adding one to an article I remember with duplicate text, Women in philosophy. However, I had to add a new parameter, nohat, to avoid creating hatnotes in the middle of paragraphs. (It also added an odd paragraph break I can't figure out how to get rid of.) I think my implementation of nohat may have been a little messy, though, so if you're an experienced template editor, could you try to clean it up? I'm also not sure whether we ought to be including a hatnote by default anyways; excerpts seem like more of a thing for editors to know about than for readers to need to know. Overall, my experience was that implementing an excerpt was a little trickier than expected; small things like the bolding of a topic when it first gets introduced for its own article got in the way, making it hard to do a clean excerpt. Sdkb (talk) 20:03, 17 March 2020 (UTC)
@Sdkb: Thanks for contacting me! In this case, I think that maybe you were thinking it the other way round: the "normal" or at least usual way to do excerpts would be to move (merge) the content from Philosophy#Role of women into Women in philosophy and leave an excerpt of the later in the former, as I just did. Please let me know if this solves the issue, or if I got it all wrong. Cheers! Sophivorus (talk) 20:19, 17 March 2020 (UTC)
@Sophivorus: I agree that it's probably better to have it the way you switched it to; that norm should probably be documented somewhere. As for your edit, though, the overall intro to women in philosophy is way too long to excerpt into Philosophy#Role of women (see the talk page discussions, e.g. here, that have winnowed that section to an appropriate length). That's the sort of thing that I was referring to above that seems to be making applying excerpting in practical situations trickier than just a blunt insertion. Sdkb (talk) 20:42, 17 March 2020 (UTC)
I just restored the winnowed version, now excerpted, at Philosophy#Role of women, but now there seems to be a paragraph break issue, where it's not properly inserting the break. It seems you have your work cut out for you in terms of issues to fix if you end up going through with the grant. (I'm sure the process of excerpting will get easier as bugs are fixed, shortcuts are introduced, and editors get used to the process. For now, this is a lot of work just to get something to the state it was already in.) Sdkb (talk) 20:52, 17 March 2020 (UTC)
@Sdkb: I couldn't spot the line break issue, not even in the article history. I guess you fixed it before publishing? Too bad you had so many problems implementing your first excerpt, I'll take it as a confirmation that the documentation and the template can indeed be improved significantly. As a volunteer I may do so eventually, but if I get the grant I'll do so faster an certainly. Finally, in case you haven't seen it yet, I think you may find the Spanish version of the Philosophy article (es:Filosofía) an interesting case study regarding the possibilities of excerpts. Cheers! Sophivorus (talk) 22:33, 17 March 2020 (UTC)
@Sophivorus: the line break issue is still present, at least on my browser and my mobile device (Android). The issue is that, whereas normally separate paragraphs have a slightly larger line break between them than between lines in the same paragraph, here the break is the same as it is within paragraphs. So they're a little smushed together. Is that true for you as well? Sdkb (talk) 23:38, 17 March 2020 (UTC)
@Sdkb: Ah yes, I see what you mean now. Apparently this is because when excerpting fragments, the <p> tags around paragraphs are not generated, so the fragments are just wrapped with <div> tags. It's a subtle issue but an issue nonetheless. I'll think about it and see if I can come up with a solution. Thanks for pointing that out!
As an afterthought, maybe a way to make excerpts more flexible while keeping them simple would be to add a parameter to control which paragraphs to excerpt, so for example {{Excerpt|Women in philosophy|paragraphs=1,3}} would excerpt paragraphs 1 and 3 from the article. Do you agree this is the way to go? Sophivorus (talk) 12:46, 20 March 2020 (UTC)
@Sdkb: Good news! It seems that the Module:Excerpt already supported cherry-picking paragraphs so adding the functionality to the template was easy. So I went ahead, added the functionality and then changed the excerpt at Philosophy#Role of women to cherry-pick the desired paragraphs (diff). The new excerpt includes one more sentence than the one you had before, but it looks like a minor thing that we can just ignore or fix by changing the intro of Women in philosophy. However, bear in mind that if the paragraph count or order changes, the excerpt may change too and without notice, so I'll leave it up to you to decide if this is an improvement in this case, or if you'd like to revert to fragments. However, overall, I think this is an improvement to excerpts. Let me know your thoughts!
PS: I just noticed you're editing the 2019–20 coronavirus pandemic article quite a bit. I think maybe you want to check out the Spanish version of the article (es:Pandemia de enfermedad por coronavirus de 2019-2020). It's much less developed in general, but it makes heavy use of excerpts (see also the corresponding excerpt tree) so you may find it an interesting case-study for improving the English version. Sophivorus (talk) 13:54, 20 March 2020 (UTC)
@Sophivorus: I definitely think the paragraph feature is a big improvement! Although the line break issue with the fragment tool should still be fixed, since sometimes it'll be more necessary to cut in the middle of a paragraph than it is here. Regarding 2019–20 coronavirus pandemic, if you're seeking to raise the profile of excerpts (as I assume you are), adding one or more to that page would be an excellent way to do it, since it's getting a gargantuan amount of editing traffic right now. I'm sure a bunch of editors would go "woah, what's that" and then "ooh, that's cool!" Just be aware that changes to that page are a massive flood right now, and editing it can be like drinking from a firehose. And it's getting quite long, so keeping down the length in sections that have their own page (e.g. response in the United States) is important. But if you find a suitable place to insert an excerpt where it'd improve the existing text, please go for it! Sdkb (talk) 17:48, 20 March 2020 (UTC)
@Sdkb:Done, let's see what happens, cheers! Sophivorus (talk) 19:46, 20 March 2020 (UTC)
As for the hatnote, do you have any thoughts on that? I can understand its use as similar to the further information hatnote, but when it's only fragments excerpted, its use seems to be inappropriate for WP:HATNOTE. And does my implementation of nohat look alright to you? Sdkb (talk) 20:42, 17 March 2020 (UTC)
@Sdkb: Having a "nohat" parameter can only help, never hurt, so I'm all for it. In fact we have a similar thing on the Spanish Wikipedia. Your implementation was perfect (I did a very minor improvement). Thanks! Sophivorus (talk) 22:33, 17 March 2020 (UTC)
A thought that occurred to me about the recently introduced paragraphs parameter: what happens when a source article changes, so that the third paragraph is no longer the third paragraph? Is that something we'd need to have a bot monitoring to fix? Sdkb (talk) 00:05, 25 March 2020 (UTC)
Update: As it happens, I had occasion to reorder some paragraphs just now, and can confirm it broke an excerpt (without alerting the breaker). So this needs some sort of solution. Sdkb (talk) 07:43, 25 March 2020 (UTC)
@Sdkb: Yea, that's the problem I was referring to when I said "bear in mind that if the paragraph count or order changes, the excerpt may change too and without notice, so I'll leave it up to you to decide if this is an improvement in this case, or if you'd like to revert to fragments". Writing a bot sounds like overkill and in any case I wouldn't know where to start (if I get an idea I'll let you know). I think the most realistic solution is to discourage using the paragraph parameter except for transcluding only the first paragraph in cases where you need a very concise intro. Transcluding paragraphs 1 and 3 for example, is asking for trouble. In my experience (based on my work in the Spanish Wikipedia), if a lead section is too long for an excerpt (like Women in philosophy is for Philosophy#Role of women) then probably the lead section is simply too long and needs to be shortened before doing the excerpt. Sophivorus (talk) 14:19, 25 March 2020 (UTC)
This is proving to be a major challenge in practical terms trying to implement excerpts on coronavirus articles. Someone (not sure who) is regularly moving all references to be defined in the reference section. For implementing excerpts on a wider scale, I think you're definitely going to have to find a way to automatically handle references like these without creating an error. Sdkb (talk) 11:02, 25 March 2020 (UTC)
This may be possible with a significant enhancement of the Module:Excerpt. Sophivorus (talk) 17:58, 3 April 2020 (UTC)
What happens when fragment tags get accidentally removed
Another issue I'm encountering when trying to use excerpts in the wild: clumsy edits can accidentally remove fragment tags, as just happened here, which then apparently broke the U.S. article. Like I said above, I'd love to see these starting working, but there's a lot of work left to do to get them to a point where they're less cumbersome to implement/maintain. Sdkb (talk) 19:24, 26 March 2020 (UTC)
@Sdkb: This is actually detectable and I just improved the template to do so. Now, every time a fragment tag is changed or removed, or a section name is changed, and consequently an excerpt breaks, the article with the broken excerpt will be added to Category:Articles with broken excerpts. Thus, you can add that category to your watchlist to be notified when an excerpt breaks. Cheers! Sophivorus (talk) 02:54, 27 March 2020 (UTC)
Nice, glad to see it can be monitored! Given how severe the disruption from a broken excerpt can be, the ideal would be for the edit window to notify you before you save it, kinda like happens with an edit conflict. That might be too much to ask though. Sdkb (talk) 05:17, 27 March 2020 (UTC)
@Sdkb: Unfortunately I had to remove the detection of broken fragments because of this issue and another issue reported in the Spanish Wikipedia that would have eventually caused trouble here too. Sooner or later I'll be able to restore the broken fragment detection functionality, but it'll require some non-trivial enhancement to the Module:Excerpt, so for now we'll have to do without. However, if a section changes name and an excerpt breaks, it'll still be added to Category:Articles with broken excerpts. Sophivorus (talk) 17:49, 3 April 2020 (UTC)
With the latest changes to the Excerpt module, broken fragment detection is back and without the previous issues. Sophivorus (talk) 22:09, 7 May 2020 (UTC)
I propose that we remove the link to WP:Excerpts when a hatnote is used with this template. The main rationale is WP:READER — the excerpts page is a Wikipedia page that will have no use to the average reader, and we shouldn't be prominently linking to it from every article that uses an excerpt. {{u|Sdkb}}talk 08:09, 5 April 2020 (UTC)
I'm going to go ahead and remove this. {{u|Sdkb}}talk 21:14, 21 April 2020 (UTC)
Formatting issue when placing <section> tags in the middle of tables
FYI: I spotted a formatting issue when placing a <section end> tag in the middle of a table to transclude only the first few rows. The section tags I am referring to are currently on 2020 Libertarian Party presidential primaries#Declared candidates and 2020 Green Party presidential primaries#Declared candidates, as it attempts to tranclude only a portion of those tables onto 2020 United States presidential election. The bug was really noticeable on the Mobile version, because it screwed up the automatic collapsible sections that are only used on mobile. This led me to examine the CSS and those collapsible <div> blocks. I now believe the bug was caused because of how <div class="excerpt-block"> and <div class="excerpt"> are added by this template, and the fact that the <section begin> tags were placed before the table started. This likely resulted in a non-complaint nesting order of <div><div><table> ... </div></div></table> -- and thus browser confusion how to interpret the div block to collapse. I apparently fixed the bug for now with this edit by adding more closing </div></div> tags. Cheers. Zzyzx11 (talk) 09:12, 16 April 2020 (UTC)
Trying to use {{Excerpt|2020 coronavirus pandemic in France}} isn't currently working. Any idea what's causing that? {{u|Sdkb}}talk 20:56, 21 April 2020 (UTC)
@Sdkb: Ah, silly me, I thought the France section at 2020 coronavirus pandemic in Europe was an excerpt. Yea I was able to reproduce the issue, thanks for the report. I'm not sure what the cause is yet, but I suspect it's related to the <noinclude> and <includeonly> tags in the lead. I'm taking note of this issue, thanks again! Sophivorus (talk) 00:59, 22 April 2020 (UTC)
@Sophivorus: I somehow hadn't noticed the edit button previously, but I like it a lot, since it does a good job of addressing the "how do I edit this content then?!" objection to excerpts. I'm still somewhat inclined to use |nohat=, but the button is a good argument against doing so. I removed the history button, though, since that's a bit much per WP:READER. One thing that would help a lot is if, when excerpting a section, the button took you to edit that section directly, rather than the whole page. Any ideas on how to implement that? {{u|Sdkb}}talk 21:56, 21 April 2020 (UTC)
@Sdkb: Hi! I agree that the history button is better left off. Someone else added it, not sure why, and I just didn't feel the need to remove it, though I didn't fully agree with it. Regarding being able to edit the section instead of the general article, unfortunately sections are identified by their number when building the edit link (https://en.wikipedia.org/w/index.php?title=2019%E2%80%9320_coronavirus_pandemic&action=edit§ion=1), but when excerpting a section, we ask for the section name ({{Excerpt|2019-2020 coronavirus pandemic|Epidemiology}}). It would be possible to add yet another parameter to the template to specify the section number, but I think that few users will understand it and even less will use it. I think the real solution would be to calculate the section number via the Module:Excerpt and build the hatnote from the module, which is doable, but rather time consuming. It's the kind of thing for which the grant would be great. Cheers! Sophivorus (talk) 00:57, 22 April 2020 (UTC)
Editors interested in this template may find discussions at Module talk:Excerpt or its archive helpful. Certes (talk) 23:30, 23 April 2020 (UTC)
@Certes: Since this template is just a straight implementation of that module, the two talk pages should be merged. {{u|Sdkb}}talk 03:50, 24 April 2020 (UTC)
@Certes and Sdkb: Agree, should be merged. Honestly I'm not too familiar with the process here at the English Wikipedia. Can I be lazy just this time and ask for someone else to do it, pretty please? Sophivorus (talk) 18:46, 29 April 2020 (UTC)
Can we figure out what to do with {{Transcluded section}} and {{Transcluding}}? At the very least, they should have big bold notices at the top saying "hey, you may be looking for or would be better served by {{Excerpt}}, and at most we may want to deprecate them. {{u|Sdkb}}talk 03:11, 25 April 2020 (UTC)
We should notify potential users of the alternatives, but I think {{Transcluded section}} and {{Transcluding}} will always have uses. Where it works, simple transclusion is an easier and cheaper option than {{Excerpt}}, and there will always be a few pages whose syntax is too arcane for Module:Excerpt to handle. Certes (talk) 10:08, 25 April 2020 (UTC)
@Sdkb: Merging templates into more general and flexible ones is a natural and desirable process with many benefits. I think that {{Excerpt}} could, in principle, be made more general and flexible to allow for hatnotes-only, plain transclusions, etc. Manually replacing the ~2000 calls to those templates for appropriate calls to {{Excerpt}} was (and is) one of the goals of my grant request. In the meantime, I added a link to {{Excerpt}} to the documentation of those templates, but there's a big difference between doing that and slapping a big bold notice at the top saying: "deprecated, please use excerpts whenever possible". I think that could happen, but would need to be proposed on the talk pages of those templates. Should we try? Sophivorus (talk) 18:44, 29 April 2020 (UTC)
@Sophivorus: perhaps wait a bit. There might be some opposition currently, but as this template improves over time, I'd expect fewer and fewer objections. An re the general concept of consolidation, 100%. I've recently proposed some of that, and may get to improving WP:CONSOLIDATION to make it more generalized at some point. {{u|Sdkb}}talk 18:50, 29 April 2020 (UTC)
Noinclude tags are getting removed every time someone uses the VisualEditor, messing up proper transclusion
See the Phabricator task I started here. Noinclude tags are necessary with excerpts to allow things like saying "the pandemic" instead of "the COVID-19 pandemic" when a page is excerpted onto COVID-19 pandemic, but this issue will need to get resolved to make their use viable outside of the short term. {{u|Sdkb}}talk 19:06, 7 May 2020 (UTC)
@Sdkb: I don't use VE but is it possible that it tries to help by simplifying piped links in a way incompatible with transclusion? For example, does it replace [[Foo bar|Foo bar]] by [[Foo bar]] after preprocessing the text in the context of the page being saved, so [[Foo bar|Foo<noinclude> bar</noinclude>]] preprocesses to [[Foo bar|Foo bar]] and gets simplified to [[Foo bar]], with no thought for what might happen if it were transcluded? Have you seen VE removing the tags in any other circumstances? Certes (talk) 19:20, 7 May 2020 (UTC)
I changed it to excerpt a fragment, which requires section tags on the other article, and that worked, but it's still not clear why the formatting to excerpt a section {{excerpt|North Carolina Supreme Court|Current Justices}} didn't work. Reywas92Talk 01:46, 23 April 2020 (UTC)
I found another case of a table not transcluding, see Template:Excerpt/testcases#Table 2. Not sure what's going on yet, since the table seems perfectly normal. I will look into both these issues soon. Sophivorus (talk) 11:46, 16 May 2020 (UTC)
@Sophivorus: The problem may be in parse() which reads
422 if token then -- found a template which is not the prefix to a line of text
424 if leadStart then -- lead has already started, so keep the template within the text, unless it's a whole line (navbox etc.)
425 if not filesOnly and not startLine then t = t .. token end
427 else [do clever things with tables]
450 else -- the next token in text is not a template
token=true (found a {} pair) and leadStart=true (lead has already started), so we execute line 425, which does nothing because startLine=true (tables start in column 1). The else blocks following lines 427 nor 450 do nothing either, because their if conditions were true. Would you want to include tables found at the start of an excerpt? If so then perhaps the table handling at line 431 should be moved so it is not conditional on leadStart being true. Certes (talk) 12:41, 16 May 2020 (UTC)
Replacing wantedBlockTemplates and unwantedInlineTemplates for templatesBlacklist
I'm thinking that replacing wantedBlockTemplates and unwantedInlineTemplates for a simple and straightforward templatesBlacklist would be easier to understand for new users and developers, and to implement in this wiki and others, since we would be able to get a good start at building a complete list of unwanted templates from categories such as Category:Article_message_templates that every wiki has. What do you think? Sophivorus (talk) 19:59, 17 May 2020 (UTC)
Theoretically, yes. Other templates such as Category:hatnote templates are also unwanted and are currently detected by their typical position as blocks before the text. If it is practical to maintain a list of all such templates (and redirects to them) then that could be a big step forward. Unfortunately, although it is easy to display a category's members to the reader, it is believed impossible to read a category listing into a Lua variable. Certes (talk) 21:10, 17 May 2020 (UTC)
I agree it would be optimal to simply list the categories of unwanted templates and have the module do the rest, and I agree that such functionality is currently not possible. My proposal was to manually build a list of all the unwanted templates, based on the current unwantedInlineTemplates and on the categories for guidance. It will be a long list (though maybe not too long), but at least it can be made mostly complete in a single day, unlike the current wantedBlockTemplates solution that will keep on requiring edits and causing requests and confusion for who knows how long. Sophivorus (talk) 21:30, 17 May 2020 (UTC)
I did a first attempt to implement this idea, see this diff and the new Module:Excerpt/i18n/sandbox. It seems to be working fine, but I only covered amboxes and hatnotes. There's also infoboxes and navboxes and probably others ahead, but building the lists is actually quite easy, I can do most of them in a day. What do you think? Sophivorus (talk) 00:41, 19 May 2020 (UTC)
In theory it looks great but I have a few concerns. Will we catch everything, including redirects? Will the list stay up-to-date? Will it be reasonably efficient? matchAny() was designed for tens of patterns and I didn't time it with thousands. We may need to do something like split the blacklist by first letter and run matchAny(t, "^{{%s*", d.templatesBlacklist[initial], "%s*%f[|}]"). Certes (talk) 01:03, 19 May 2020 (UTC)
Hmm, good observations. This is getting rather complicated. Maybe the current solution is good enough for now. XD If I come up with something I'll let you know. Thanks again for your insights! Sophivorus (talk) 01:47, 19 May 2020 (UTC)
Hi again! Today I did a first implementation of the functionality that transcludes references defined outside of the transcluded lead or section (diff). See the testcase at Template:Excerpt/testcases#Undefined reference. Tomorrow I'll test it in other contexts and refine it. As always, any feedback and extra testing is welcome! Sophivorus (talk) 20:11, 19 May 2020 (UTC)
Add templates to wantedBlockTemplates
This edit request has been answered. Set the |answered= or |ans= parameter to no to reactivate your request.
@Pppery: Apologies for this, due to my global permission, I see the "Edit" button rather than "View source" and I have to notice that little lock icon which I don't always do. I need to add the following templates to the wantedBlockTemplates variable at Module:Excerpt/i18n:
[Tt]ree chart[^|}]*
[Cc]hart[^|}]*
[Tt]elevision ratings graph
Due to the nature of the variable, several more similar requests will be required over time. Sophivorus (talk) 14:33, 16 May 2020 (UTC)
Today I noticed that this regex was also removing "groups" from ref tags. After some thought, rather than fix it, I thought I should keep that behavior and I even refined it. That's because when transcluding references with groups, often the transcluding article will display a reference error that requires to add an extra {{Reflist}}. While this may be ok in theory, I think in practice causes more confusion than benefits, so I think it's better to just remove groups and have the references merge with the main group. However, if someone disagrees, I'm willing to add some kind of option to keep groups, or change the default behavior. Let me know! Cheers, Sophivorus (talk) 14:23, 20 May 2020 (UTC)
Fragment transclusion
The module already handles onlyinclude.../onlyinclude. Once the fragment name has been determined, it should be possible to parse section begin...section end similarly. (I'm calling this feature "fragment transclusion" to distinguish it from section transclusion with the Article#Section syntax, which is already implemented.) Certes (talk) 20:07, 5 April 2020 (UTC)
@Certes: Hi! Following your suggestion, I made some changes to the sandbox that add basic support for fragment transclusion. I made some tests too, to illustrate what works and what doesn't. Simple fragments work. However, if an article has two or more <sections> with the same name, the module transcludes only the first, whereas #lst transcludes all of them. Finally, if a section name contains any character that has special meaning in regular expressions (notably "-") then the transclusion fails. PS, I also generalized the parameter of the cleanupText method, so we should keep that in mind to prevent this issue from happening again. Cheers! Sophivorus (talk) 22:43, 18 April 2020 (UTC)
@Sophivorus: That looks like a good idea but, as you say, we need to change a couple of things. Firstly, previewing a gallery like Portal:Transport with the sandbox gives a Lua error in Module:Excerpt at line 494: attempt to index local 'options' (a boolean value). My recent change to Module:Random slideshow makes it pass keepSubsections=True to cleanupText(), but the sandbox version now needs a table, not a boolean. Secondly, the line local fragment = mw.ustring.match(...) is only going to match one section. To match more you'll need to do something more complicated: either do what I did with onlyinclude near the start of cleanupText, or call match() repeatedly with init set to the previous position, which you can capture with (), until it fails to match. Hope that helps, Certes (talk) 00:11, 19 April 2020 (UTC)
@Certes: Added support for multiple fragments and for special characters, see testcases. Thanks for your hints, they were really useful! Sophivorus (talk) 13:20, 19 April 2020 (UTC)
@Certes: Hi! I'd really like to update the module to the latest sandbox version with the new fragment transclusion functionality, but I keep finding bugs. Now it's Template:Excerpt/testcases#Complicated fragment, which is a copy-paste of a real use case at 2020 United States presidential election#Libertarian Party nomination. I tried replacing the fragment transclusion code for a plain call to #lst within the module (diff) but it didn't help with the bug so I reverted back. It would be great to fix this issue, like any other, but maybe such monstrosities should never exist in the first place and we're better off replacing such calls to {{Excerpt}} for plain #lst calls? What do you think? Thanks for any help! Sophivorus (talk) 18:20, 29 April 2020 (UTC)
@Sophivorus: Something is adding an image File:Jacob Hornberger by Gage Skidmore (cropped) (2).jpg at the start of the output, immediately before the "{|" which starts the table. There needs to be a line feed in between, because "{|" only works at the start of a line. I tried adding the temporary workaround text=mw.ustring.gsub(text, "{", "\n{", 1) just before the return statement at the end of main(), and the table appeared correctly. So, whatever adds the image should probably also add a "\n" at the end of it. By the way, there is also a missing %s* in "section%s+end%s*=", which isn't the problem here. Certes (talk) 19:01, 29 April 2020 (UTC)
@Certes: Thanks for the quick reply and the sharp eye. I replaced those two excerpts for #lst calls because I think that fixing them is beyond what I'm willing. However, after doing that, I tested the sandbox version on some other articles and found that Results of the 2020 Republican Party presidential primaries#Alabama was also causing trouble, so I restored the version of the sandbox that calls the #lst parser function to get the fragment (rather than the custom made code) and at least that bug was gone. So finally I updated the module to the sandbox version. I think the current situation is far from ideal, but it never will be and in any case I'd like to move forward with other tasks. To me, this only confirms that fragments and those abominable <section> tags should be sent back to the fiery pit from whence they came. PS, I also updated Module:Random slideshow to the new syntax (diff). Sophivorus (talk) 22:55, 29 April 2020 (UTC)
Apparently calling #lst from within the module somehow prevents some of the later processing of wikitext, as can be seen from, Module:Excerpt/testcases#Simple where references are retained even though keepRefs is not enabled. This suggests that recovering the custom made parsing of <section> is the way to go. Sophivorus (talk) 15:50, 20 May 2020 (UTC)
Failed excerpt
See Module:Excerpt/testcases#Basic 2. I'm not sure about the cause yet, but I bet it's related to the <noinclude> and <includeonly> tags all over the lead. Sophivorus (talk) 01:34, 22 April 2020 (UTC)
A transcluding page only sees one line consisting of an #ifeq section. Module:Excerpt regards this as a template on a line by itself at the start of an article, and discards it as if it were a hatnote. The relevant check is if leadstart then within parse(). This module was designed for displaying the lead of a simple, normal article. Writing a bulletproof wikitext parser which handles all permutations of nested transclusion and conditional text is way beyond what we ever dreamed of it doing, and I don't think it's feasible in a Lua module which potentially has to prepare 100 articles for a portal gallery in 10 seconds. Please feel free to prove me wrong! Certes (talk) 06:38, 22 April 2020 (UTC)
@Certes: Thanks for the investigation! I added a regex for parser functions to wantedBlockTemplates and it seems to have fixed this issue. I think it'd be better to detect parser functions from the module rather than having to add them to wantedBlockTemplates, but I don't see the proper way to code it yet, so I did that for now. Sophivorus (talk) 18:23, 28 April 2020 (UTC)
Found the proper way, now parser functions are not confused for block templates and are kept by default. Sophivorus (talk) 17:05, 20 May 2020 (UTC)
filesOnly
I did a small change that adds filesOnly as another option available for use to templates, see the testcases at Module:Excerpt/testcases#Files only and Template:Excerpt/testcases#Files only. Seems to be working wonderfully. I think this functionality, as well as the upcoming tablesOnly and listsOnly options, will prove exceedingly useful to reuse content without resorting to those cursed <section> tags. PS, Module:Random_slideshow will need to be updated to the new syntax when this change goes live. Cheers! Sophivorus (talk) 18:10, 20 May 2020 (UTC)
Hi yet again! While working on #filesOnly, #tablesOnly, #listsOnly and the split between the main module and the /templates submodule, I slowly came to realize that this module is becoming a transclusion engine, that is, a module able to transclude any element of any page that can be specified. I think taking this approach explicitly could make this module extremely useful not just for the current wrapper templates, but also for many future and still unknown uses, and would also help us orient development towards a common and clearly defined goal. I know in my case, it's already helping. What do you think? Sophivorus (talk) 18:31, 22 May 2020 (UTC)
Moved from Template talk:Transclude lead excerpt
Done in Module:Excerpt. @Armanaziz: thanks, good catch. Certes (talk) 18:50, 6 May 2018 (UTC)
Fixed. Thanks for the report. The nested apostrophe parser now handles partially italic titles in bold, such as '''SS ''Vienna''''' and '''''Billboard'' 200'''. I've also tweaked the image parsing so that it displays the ship image in preference to the U.S. flag. Certes (talk) 12:20, 22 May 2018 (UTC)
Comment: It seems to be due to the way that this template is recognizing the wikitable. It seems to see each row of the wikitable is seen as a paragraph. Dreamy Jazz (talk) 20:19, 29 May 2018 (UTC)
.svg){kind=link}
You are invited to join the discussion at Wikipedia_talk:Short_description#Short_description_broken_at_COVID-19_pandemic_and_possibly_elsewhere. {{u|Sdkb}} talk 21:32, 14 May 2020 (UTC)
_(2).jpg){kind=link}