|
|
You're in the
sql server full text search
group:Integration of Lucene.Net and SQL Server 2005?
sql server full text search:
I am looking for an alternative to the SQL Server Fulltext Engine since it is just not fast enough and does not offer the sorting I require. Also, my customer wants a "did you mean google" when "gogle" was entered feature. From what I have seen all that is possible with Lucene.Net. I am planning to export a table of about 3 Million keywords to Lucene.Net and of course I have to implement a way to keep that index up to date (a latency of one hour is allowed). Also, I would still like to initiate searching through stored procs. Ideally a table-value function with CLR integration would get a list of matching ID's from Lucene. I wanted to check if maybe someone has used such approach or heard about something similar, or if there are general restrictions that make the success of this approach unlikely. TIA for any comments, Regards DC
I did this. It works quite well. SQL FTS is faster at indexing and you have
to jump through hoops to get Lucene working correctly, but properly done it is an alternative, especially when you need to do filtering. -- RelevantNoise.com - dedicated to mining blogs for business intelligence. Looking for a SQL Server replication book? http://www.nwsu.com/0974973602.html Looking for a FAQ on Indexing Services/SQL FTS http://www.indexserverfaq.com [quoted text, click to view] "DC" <dc@upsize.de> wrote in message news:1189508864.757520.62150@57g2000hsv.googlegroups.com... > Hi, > > I am looking for an alternative to the SQL Server Fulltext Engine > since it is just not fast enough and does not offer the sorting I > require. Also, my customer wants a "did you mean google" when "gogle" > was entered feature. From what I have seen all that is possible with > Lucene.Net. > > I am planning to export a table of about 3 Million keywords to > Lucene.Net and of course I have to implement a way to keep that index > up to date (a latency of one hour is allowed). Also, I would still > like to initiate searching through stored procs. Ideally a table-value > function with CLR integration would get a list of matching ID's from > Lucene. > > I wanted to check if maybe someone has used such approach or heard > about something similar, or if there are general restrictions that > make the success of this approach unlikely. > > TIA for any comments, > > Regards > DC >
I am writing an article on it for simple-talk.
-- RelevantNoise.com - dedicated to mining blogs for business intelligence. Looking for a SQL Server replication book? http://www.nwsu.com/0974973602.html Looking for a FAQ on Indexing Services/SQL FTS http://www.indexserverfaq.com [quoted text, click to view] "DC" <dc@upsize.de> wrote in message news:1189525577.389287.163260@57g2000hsv.googlegroups.com... > On 11 Sep., 16:12, "Hilary Cotter" <hilary.cot...@gmail.com> wrote: >> I did this. It works quite well. SQL FTS is faster at indexing and you >> have >> to jump through hoops to get Lucene working correctly, but properly done >> it >> is an alternative, especially when you need to do filtering. >> >> -- >> RelevantNoise.com - dedicated to mining blogs for business intelligence. >> >> Looking for a SQL Server replication >> book?http://www.nwsu.com/0974973602.html >> >> Looking for a FAQ on Indexing Services/SQL >> FTShttp://www.indexserverfaq.com"DC" <d...@upsize.de> wrote in message >> >> news:1189508864.757520.62150@57g2000hsv.googlegroups.com... >> >> >> >> > Hi, >> >> > I am looking for an alternative to the SQL Server Fulltext Engine >> > since it is just not fast enough and does not offer the sorting I >> > require. Also, my customer wants a "did you mean google" when "gogle" >> > was entered feature. From what I have seen all that is possible with >> > Lucene.Net. >> >> > I am planning to export a table of about 3 Million keywords to >> > Lucene.Net and of course I have to implement a way to keep that index >> > up to date (a latency of one hour is allowed). Also, I would still >> > like to initiate searching through stored procs. Ideally a table-value >> > function with CLR integration would get a list of matching ID's from >> > Lucene. >> >> > I wanted to check if maybe someone has used such approach or heard >> > about something similar, or if there are general restrictions that >> > make the success of this approach unlikely. >> >> > TIA for any comments, >> >> > Regards >> > DC- Zitierten Text ausblenden - >> >> - Zitierten Text anzeigen - > > Thank you for letting my know, Hilary! That's encouraging. Do you > consider designing an "SQL Server / Lucene.Net Bridge" product? >
[quoted text, click to view]
On 11 Sep., 16:12, "Hilary Cotter" <hilary.cot...@gmail.com> wrote: > I did this. It works quite well. SQL FTS is faster at indexing and you have > to jump through hoops to get Lucene working correctly, but properly done it > is an alternative, especially when you need to do filtering. > > -- > RelevantNoise.com - dedicated to mining blogs for business intelligence. > > Looking for a SQL Server replication book?http://www.nwsu.com/0974973602.html > > Looking for a FAQ on Indexing Services/SQL FTShttp://www.indexserverfaq.com"DC" <d...@upsize.de> wrote in message > > news:1189508864.757520.62150@57g2000hsv.googlegroups.com... > > > > > Hi, > > > I am looking for an alternative to the SQL Server Fulltext Engine > > since it is just not fast enough and does not offer the sorting I > > require. Also, my customer wants a "did you mean google" when "gogle" > > was entered feature. From what I have seen all that is possible with > > Lucene.Net. > > > I am planning to export a table of about 3 Million keywords to > > Lucene.Net and of course I have to implement a way to keep that index > > up to date (a latency of one hour is allowed). Also, I would still > > like to initiate searching through stored procs. Ideally a table-value > > function with CLR integration would get a list of matching ID's from > > Lucene. > > > I wanted to check if maybe someone has used such approach or heard > > about something similar, or if there are general restrictions that > > make the success of this approach unlikely. > > > TIA for any comments, > > > Regards > > DC- Zitierten Text ausblenden - > > - Zitierten Text anzeigen - Thank you for letting my know, Hilary! That's encouraging. Do you consider designing an "SQL Server / Lucene.Net Bridge" product?
Here is the code. The field I am indexing and searching is Contents.
using System.Data; using System.Data.SqlClient; using System.Data.SqlTypes; using System.Collections; using Microsoft.SqlServer.Server; using Lucene.Net; using Lucene.Net.Search; using Lucene.Net.Documents; using Lucene.Net.QueryParsers; using Lucene.Net.Analysis; using Lucene.Net.Analysis.Standard; public partial class UserDefinedFunctions { [Microsoft.SqlServer.Server.SqlFunction(DataAccess = DataAccessKind.Read, FillRowMethodName = "FillRow", SystemDataAccess = SystemDataAccessKind.Read, TableDefinition = "PK int")] public static IEnumerable LuceneSearch(string SearchPhrase) { Searchable[] remoteSearch = new Searchable[7]; remoteSearch[0] = new IndexSearcher("//10.0.0.1/14"); remoteSearch[1] = new IndexSearcher("//10.0.0.2/15"); remoteSearch[2] = new IndexSearcher("//10.0.0.3/16"); remoteSearch[3] = new IndexSearcher("//10.0.0.4/17"); remoteSearch[4] = new IndexSearcher("//10.0.0.4/18"); remoteSearch[5] = new IndexSearcher("//10.0.0.4/18a"); remoteSearch[6] = new IndexSearcher("//10.0.0.4/19"); ParallelMultiSearcher searcher = new ParallelMultiSearcher(remoteSearch); QueryParser qp = new QueryParser("Contents", new Lucene.Net.Analysis.Standard.StandardAnalyzer()); Query query = qp.Parse(SearchPhrase); Hits hits = searcher.Search(query); int[] PKs = new int[hits.Length()]; for (int i = 0; i < hits.Length(); i++) { Document doc = hits.Doc(i); PKs[i] = int.Parse(doc.Get("PK")); } return postids ; } public static void FillRow(object row, out int PK) { PK= (int)row ; } }; -- RelevantNoise.com - dedicated to mining blogs for business intelligence. Looking for a SQL Server replication book? http://www.nwsu.com/0974973602.html Looking for a FAQ on Indexing Services/SQL FTS http://www.indexserverfaq.com [quoted text, click to view] "DC" <dc@upsize.de> wrote in message news:1189581463.912693.245440@g4g2000hsf.googlegroups.com... > On 11 Sep., 19:22, "Hilary Cotter" <hilary.cot...@gmail.com> wrote: >> I am writing an article on it for simple-talk. >> >> -- >> RelevantNoise.com - dedicated to mining blogs for business intelligence. >> >> Looking for a SQL Server replication >> book?http://www.nwsu.com/0974973602.html >> >> Looking for a FAQ on Indexing Services/SQL >> FTShttp://www.indexserverfaq.com"DC" <d...@upsize.de> wrote in message >> >> news:1189525577.389287.163260@57g2000hsv.googlegroups.com... >> >> >> >> > On 11 Sep., 16:12, "Hilary Cotter" <hilary.cot...@gmail.com> wrote: >> >> I did this. It works quite well. SQL FTS is faster at indexing and you >> >> have >> >> to jump through hoops to get Lucene working correctly, but properly >> >> done >> >> it >> >> is an alternative, especially when you need to do filtering. >> >> >> -- >> >> RelevantNoise.com - dedicated to mining blogs for business >> >> intelligence. >> >> >> Looking for a SQL Server replication >> >> book?http://www.nwsu.com/0974973602.html >> >> >> Looking for a FAQ on Indexing Services/SQL >> >> FTShttp://www.indexserverfaq.com"DC" <d...@upsize.de> wrote in message >> >> >>news:1189508864.757520.62150@57g2000hsv.googlegroups.com... >> >> >> > Hi, >> >> >> > I am looking for an alternative to the SQL Server Fulltext Engine >> >> > since it is just not fast enough and does not offer the sorting I >> >> > require. Also, my customer wants a "did you mean google" when >> >> > "gogle" >> >> > was entered feature. From what I have seen all that is possible with >> >> > Lucene.Net. >> >> >> > I am planning to export a table of about 3 Million keywords to >> >> > Lucene.Net and of course I have to implement a way to keep that >> >> > index >> >> > up to date (a latency of one hour is allowed). Also, I would still >> >> > like to initiate searching through stored procs. Ideally a >> >> > table-value >> >> > function with CLR integration would get a list of matching ID's from >> >> > Lucene. >> >> >> > I wanted to check if maybe someone has used such approach or heard >> >> > about something similar, or if there are general restrictions that >> >> > make the success of this approach unlikely. >> >> >> > TIA for any comments, >> >> >> > Regards >> >> > DC- Zitierten Text ausblenden - >> >> >> - Zitierten Text anzeigen - >> >> > Thank you for letting my know, Hilary! That's encouraging. Do you >> > consider designing an "SQL Server / Lucene.Net Bridge" product?- >> > Zitierten Text ausblenden - >> >> - Zitierten Text anzeigen - > > Thank you! I will wait for that article before I do anything. > > Regards > DC >
[quoted text, click to view]
On 11 Sep., 19:22, "Hilary Cotter" <hilary.cot...@gmail.com> wrote: > I am writing an article on it for simple-talk. > > -- > RelevantNoise.com - dedicated to mining blogs for business intelligence. > > Looking for a SQL Server replication book?http://www.nwsu.com/0974973602.html > > Looking for a FAQ on Indexing Services/SQL FTShttp://www.indexserverfaq.com"DC" <d...@upsize.de> wrote in message > > news:1189525577.389287.163260@57g2000hsv.googlegroups.com... > > > > > On 11 Sep., 16:12, "Hilary Cotter" <hilary.cot...@gmail.com> wrote: > >> I did this. It works quite well. SQL FTS is faster at indexing and you > >> have > >> to jump through hoops to get Lucene working correctly, but properly done > >> it > >> is an alternative, especially when you need to do filtering. > > >> -- > >> RelevantNoise.com - dedicated to mining blogs for business intelligence. > > >> Looking for a SQL Server replication > >> book?http://www.nwsu.com/0974973602.html > > >> Looking for a FAQ on Indexing Services/SQL > >> FTShttp://www.indexserverfaq.com"DC" <d...@upsize.de> wrote in message > > >>news:1189508864.757520.62150@57g2000hsv.googlegroups.com... > > >> > Hi, > > >> > I am looking for an alternative to the SQL Server Fulltext Engine > >> > since it is just not fast enough and does not offer the sorting I > >> > require. Also, my customer wants a "did you mean google" when "gogle" > >> > was entered feature. From what I have seen all that is possible with > >> > Lucene.Net. > > >> > I am planning to export a table of about 3 Million keywords to > >> > Lucene.Net and of course I have to implement a way to keep that index > >> > up to date (a latency of one hour is allowed). Also, I would still > >> > like to initiate searching through stored procs. Ideally a table-value > >> > function with CLR integration would get a list of matching ID's from > >> > Lucene. > > >> > I wanted to check if maybe someone has used such approach or heard > >> > about something similar, or if there are general restrictions that > >> > make the success of this approach unlikely. > > >> > TIA for any comments, > > >> > Regards > >> > DC- Zitierten Text ausblenden - > > >> - Zitierten Text anzeigen - > > > Thank you for letting my know, Hilary! That's encouraging. Do you > > consider designing an "SQL Server / Lucene.Net Bridge" product?- Zitierten Text ausblenden - > > - Zitierten Text anzeigen - Thank you! I will wait for that article before I do anything. Regards DC
Don't see what you're looking for? Try a search.
|
June 2003
July 2003
August 2003
September 2003
October 2003
November 2003
December 2003
January 2004
February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
| home · blog · groups | Questions? Comments? Contact the dn |