Year of the Dragon: Through May 28th, claim free Expansion Pack (excluding Vecna Unleashed) or a Greater Elixir of Discovery! Speak to Xatheral in the Hall of Heroes.
Known Issue: Xatheral has been temporarily disabled while we investigate a bug with Year of the Dragon gift delivery.DDO Forums edit
Game mechanics • Newbie guide • In development • DDO Store • Social Media
Challenges • Classes • Collectables • Crafting • Enhancements • Epic Destinies • Favor • Feats
Glossary • Items • Maps • Monsters • Places • Quests • Races • Reincarnation • Skills • Spells
Please create an account or log in to build a reputation and unlock more editing privileges, and then visit DDO wiki's IRC Chat/Discord if you need any help!
Talk:Wiki search engine upgrade project
Jump to navigation
Jump to search
There has been some discussion on the wiki's IRC chat channel about potential upgrades to the way the wiki does searches, and I want to make sure everyone is aware of the discussions. So, here they are:
July 18, 2012
17:52:01 MeliCat heya 17:52:17 MeliCat why can't i find https://ddowiki.com/page/Greater_Shout when i type 'Greater Shout" in the front page? 18:03:20 MeliCat anyway... sorry but if the wiki gets harder and harder to find stuff it's going to become as useless as the Compendium. Please may this not happen 18:03:24 *** MeliCat quit (Read error: Connection reset by peer) juz_at_lar@Rizon-7FED8813.perm.iinet.net.au ...Snip... 20:43:11 Shoe|Baby The answer to MeliCat's question is that he did not click the checkbox for search titles only... Doing this puts Greater Shout on the top of the list when typed into search. ...Snip... 20:47:16 Trekna To be fair the average user would expect to be taken directly to a direct match ...Snip... 20:50:33 ShoeMaker That's not the way the http://www.sphinxsearch.com/ that the wiki uses works... It is no different than any other wiki that uses this search engine... 20:51:23 ShoeMaker Although, I suppose that I could see if there was a better search engine to use... That may pose a problem with the FireFox plugin that the wiki has too though... 20:53:44 Trekna Its just... Odd that searching for greater shout is actually searching for greater and shout independently 20:54:11 Sjachvorastrix Did you try putting it in quotes? 20:54:15 Trekna Yes 20:54:38 Trekna Doesn't make a difference 20:55:12 Sjachvorastrix Ok... that is weird. 20:56:02 ShoeMaker Give me a second.... 20:56:21 Sjachvorastrix And yes, if I do a search for "Greater Shout", I expect any search engine to recognize that I intend it to find the complete phrase and not give me disjointed informaiton 20:56:23 Sjachvorastrix information* 20:58:20 ShoeMaker It is an option in the advance search, I am researching if there is a way for it to be checked by default or even better, a way for the search engine to recognize '...Search for this...' or "...Search for this.." and the operators AND/OR/XOR/NOT 20:58:23 Trekna Using Marks query... It returns over 3000 results for greater and 64 for shout... Logically Greater Shout as a query could not possibly return more than 64 results.... Yet it returns 1000+ 20:58:27 Sjachvorastrix Especially since there is no link on the front page to go to an advanced search page 20:58:54 Trekna Marks = Melis....damn you autocorrect 20:59:45 ShoeMaker The results page is the advanced search page at the bottom (unless you have an account, then you can specify some defaults). 21:00:30 Trekna Save headaches by defaulting to match all not match some 21:01:31 Sjachvorastrix Ok, now I find the link to go to the advanced search page on the home page... on the opposite side of the screen from the search box >.< 21:05:02 ShoeMaker Wikipedia uses: https://www.mediawiki.org/wiki/Extension:MWSearch && https://www.mediawiki.org/wiki/Extension:OpenSearchXml && https://www.mediawiki.org/wiki/Extension:TitleKey 21:05:29 ShoeMaker I'll talk to Xevo about us switching to that... This would allow for predictive search in the search on the home screen...
From: ShoeMaker <technical_13 yahoo com>
To: Xevo --Hidden--
Sent: Wednesday, July 18, 2012 9:24 PM
Subject: Wiki search engine options...
All of the ...Snip...s are unrelated bot responses and room modes and such... I was napping when the original question was posed. So, what do you think? I did some research on it, and it looks like MW1.17 uses a beta version of Sphinx, and 1.18 uses the first stable release... But I can't seem to find the change log on it to see what has been changed... Would be great if you could come into chat to figure out the best course of possible change for this. Do we upgrade to MW1.18 and keep using Sphinx and see what is new for a while, do we change over to the search setup that wikipedia uses (I do like their predictive search bar), or is there another alternative? If I don't answer right away, you can ask my bot (Kobold_Scribe) where I am (Where is Shoe?) and he should tell you a fairly accurate answer. I check the IRC about once every hour or so when I am awake, and slightly less often when I am sleeping. :p If I'm sleeping, I will see any discussion in the chat log of my bot and can send you another email if we miss each other. :D
July 19, 2012
From: Xevo --Hidden--
To: ShoeMaker <technical_13 yahoo com>
Sent: Thursday, July 19, 2012 10:38 PM
Subject: Wiki search engine options...
I have changed it so that it will "Search All" by default. Now when I search for Greater Shout, the expected page is the first result.
July 20, 2012
From: ShoeMaker <technical_13 yahoo com>
To: Xevo --Hidden--
Sent: Friday, July 20, 2012 11:36 AM
Subject: Wiki search engine options...
That is a start... I would still like to research the ability for predictive search like Wikipedia has... So when you start typing in "Gre"... It populates a dropdown list of everything that starts with "Gre" like "Great this", "Greater that", "Green stuffs", ... I think that would be cool. :D
Thanks,
Shoe
From: Xevo --Hidden--
To: ShoeMaker <technical_13 yahoo com>
Sent: Friday, July 20, 2012 12:03 PM
Subject: Wiki search engine options...
I have a log file that contains every search that has been placed by users. Maybe it would be best to use previous searches as the input for the predictive search. I think that would work better than trying to predict a search off of the text in all the pages.
From: Xevo --Hidden--
To: ShoeMaker <technical_13 yahoo com>
Sent: Friday, July 20, 2012 12:04 PM
Subject: Wiki search engine options...
I suppose the one downside to this approach is that if somebody searches for something irrelevant or obscene, it would come back as a suggestion ;) Maybe I could only suggest keywords that have been searched for multiple times ?
From: ShoeMaker <technical_13 yahoo com>
To: Xevo --Hidden--
Sent: Friday, July 20, 2012 12:24 PM
'Subject: Wiki search engine options...
I think that Wikipedia has their predictive search set up to match page titles, and if your search doesn't match any title, hitting [Enter] takes you the search content / advanced search. Using your idea, maybe the search log could first be matched against a list of "restricted" words, the vulgar, obscene, and known spam... Then, whatever passes that test could be logged to a MediaWiki:Predictive_Search for review and modification my any admin to eliminate irrelevant/off-the-wall stuffs and parsed as the predictive matches?
Thanks,
Shoe
July 21, 2012
18:58:00 ShoeMaker !8ball has x3v0 found a solution for the search engine yet? 18:58:01 Kobold_Scribe Kobold is certain 18:58:07 ShoeMaker Sweet. 18:58:14 ShoeMaker Kobold says you have a solution! 18:58:31 ShoeMaker My kobold magic 8ball that is. 18:59:38 x3v0 I'm not so sure about lucene 19:00:16 x3v0 it uses much more memory than sphinx search does 19:01:04 x3v0 i would probably have to run it on a separate server. i would be willing to do that if the search is a lot better, but i don't think it really is 19:01:48 ShoeMaker Can the new stable version of Sphinx do the predictive search? 19:02:02 ShoeMaker once we upgrade to 1.18 that is... 19:02:05 x3v0 i could probably get away with running it on the torwiki server though, that really isn't getting much traffic 19:02:07 ShoeMaker I couldn't find release notes. 19:02:18 ShoeMaker Mostly me and the spambots, no? 19:02:19 ShoeMaker lol 19:02:22 x3v0 heh 19:02:29 x3v0 i think i have cut down on the bots 19:02:51 x3v0 I'm running the traffic through cloud flare now 19:02:55 x3v0 which block spam bots 19:03:01 ShoeMaker Oh, nice. 19:03:18 x3v0 i may do that with ddo wiki, I'm trying it out with for now 19:03:24 x3v0 *with tor now 19:03:53 x3v0 i only saw 2 spammers last week, which is way less than normal 19:06:38 x3v0 id probably choose incapsula for ddowiki though 19:06:50 x3v0 i just think they seem more reputable. I'm not sure why 19:06:53 *** ShoeMaker is now known as Shoe|Baby 19:14:05 Trekna|Android Good number of spam edits lately 19:17:35 x3v0 oh…awesome 19:17:37 x3v0 Search Box "As-You-Type" Suggestions 19:17:37 x3v0 Set the following options in LocalSettings.php after the above require_once line: 19:17:38 x3v0 $wgEnableMWSuggest - set to true to enable MediaWiki support for search box suggestions 19:17:39 x3v0 $wgEnableSphinxPrefixSearch - set to true to return suggestions by matching the query against the beginning of page titles 19:18:02 x3v0 the new version of SphinxSearch DOES support predictive search 19:22:19 x3v0 I'm going to upgrade to 1.18 this weekend 19:23:00 x3v0 I want to go to .19, but I think it would be too risky. We would be using a lot of extensions that were not fully tested in 1.19 19:28:56 Shoe|Baby I was hoping so... 19:29:13 Shoe|Baby I think it would be safe myself 19:30:18 Trekna|Android Just happy it searches for all terms 19:30:44 Shoe|Baby lol 19:30:56 Trekna|Android instead of a mix
July 22, 2012
...Snip... 09:59:34 ShoeMaker Was the discussion about the search engine x3v0 and I had last night, and some comments on the spambots after.... For those interested. ...Snip... 10:01:19 ShoeMaker Was the rest of the conversation. 10:03:11 ShoeMaker I'll make a censored talk page about it on the wiki shortly. Snipping out the non-relevant stuffs. 10:11:49 x3v0 im working on that now 10:14:25 x3v0 shoe, are there some templates you can think of that make heavy use of certain extensions? im trying to find the best pages to look at to make sure nothing broke 10:14:51 x3v0 i have the new version installed an working on my virtual machine copy of ddowiki 10:18:37 ShoeMaker The Named_xxx templates 10:19:11 ShoeMaker They use all of the extensions directly or as a transclusion with all of the templates used in the templates on the item pages with templates. ...Snip... 10:21:05 x3v0 yea, that should be a good test ... More to come? ???
ShoeMaker (Contributions • Message) 11:26, July 22, 2012 (EDT)
I assume that the recent change where it now lists the search box, several links about the search content (content pages, multimedia, help, etc.), and then all the places it could search (main, talk, user, user talk, etc.)... and THEN actually shows the links, is a result of any changes that're being made? Is there any way to, y'know, put all that back below the actual links that I'm just not seeing, or am I stuck having to scroll past all this crap I don't care about to get to where I might be able to find what I'm actually searching for?
LrdSlvrhnd (Contributions • Message) 15:52, July 23, 2012 (EDT)
- That is part of the overall wiki upgrade from 1.17 to 1.19 and we are working to make it look better and be more user friendly...
ShoeMaker (Contributions • Message) 16:31, July 23, 2012 (EDT)
- This looks good but I don't like images showing up on the search page by default. My 2cp. Book O Dragons (Contributions • Message) 16:42, July 31, 2012 (EDT)
- You could try going to your search preferences page and put a check in all the namespace boxes that you want to be searched (obviously leaving out file) and your results would be more refined for you by default... You can also click on the advanced search link if you wanted to add some of those namespaces back in for just one search.
ShoeMaker (Contributions • Message) 09:08, August 1, 2012 (EDT)