I'm quite new to MediaWiki, and now I have a bit of a problem. I have the title of some Wiki page, and I want to get just the text of a said page using api.php, but all that I have found in the API is a way to obtain the Wiki content of the page (with wiki markup). I used this HTTP request...
/api.php?action=query&prop=revisions&rvlimit=1&rvprop=content&format=xml&titles=test
But I need only the textual content, without the Wiki markup. Is that possible with the MediaWiki API?
Python users coming to this question might be interested in the
wikipedia
module (docs):Every formatting, except for sections (
==
) is striped away.Use action=render to get the cleanest possible page:
https://wiki.eclipse.org/Tip_of_the_Day/Eclipse_Tips/Now_where_was_I?action=render
vs
https://wiki.eclipse.org/Tip_of_the_Day/Eclipse_Tips/Now_where_was_I
You can get the wiki data in text format from the API by using the
explaintext
parameter. Plus, if you need to access many titles' information, you can get all the titles' wiki data in a single call. Use the pipe character|
to separate each title. For example, this API call will return the data from both the "Google" and "Yahoo" pages:Parameters:
explaintext
: Return extracts as plain text instead of limited HTML.exlimit=max
: Return more than one result. The max is currently 20.exintro
: Return only the content before the first section. If you want the full data, just remove this.redirects=
: Resolve redirect issues.I don't think it is possible using the API to get just the text.
What has worked for me was to request the HTML page (using the normal URL that you would use in a browser) and strip out the HTML tags under the content div.
EDIT:
I have had good results using HTML Parser for Java. It has examples of how to strip out HTML tags under a given DIV.
Wiki pages without any formatting symbols wouldn't really make much sense in many cases.
You can strip out the formatting yourself, if you want, but you'll break some stuff in the process.
(Unless you are creating something like a search engine, in which case you'll only need the text parts and can ignore formatting symbols completely)
The TextExtracts extension of the API does about what you're asking. Use
prop=extracts
to get a cleaned up response. For example, this link will give you cleaned up text for the Stack Overflow article. What's also nice is that it still includes section tags, so you can identify individual sections of the article.Just to include a visible link in my answer, the above link looks like:
Edit: As Amr mentioned, TextExtracts is an extension to MediaWiki, so it won't necessarily be available for every MediaWiki site.