|
|
You're in the
sql server full text search
group:SQL Server Full-Text Search Performance
sql server full text search:
We are using SQL Server 2000 and we are doing Full-Text Indexing on a table that has 5 Text Columns (we have Full Text Indexing on these 5 columns). There are around 1.5 million records in this table. And the size of the Full-Text Catalog is around 1 GB. The performance issue we are facing is some searches are slow. If I search for “to kill a mockingbird†it returns results very quickly and if I search for “numbers†the search is slower. So we are looking to increase the search speed. And also we need to get paginated results in our application. So what options I have to try? I am thinking of these options: • Upgrade to SQL Server 2005 and partition this table based on the year. Is it possible to create different Full-Text Catalogs for each partition? • I just learnt about Lucene4DB. http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . Has anyone used this tool in their application? Will there be a gain in performance in the search speed and maintaining the indexes. Thanks Vijay
Hi Hilary Cotter,
Thanks for your quick reply. I have a few followup questions: Can we create a FT Catalog for each partition in a partitioned table in SQL 2005? (If the FT Catalog is smaller then hopefully the searches will be faster) Can you point me to any sources that show the comparision of perfomance in Seraches and Indexing between Lucene4DB and SQL Server? When will Katmai (SQL 2008) release? Thanks in advance, Vijay [quoted text, click to view] "Hilary Cotter" wrote:
> SQL 2005 is a great choice. > > Lucene is also one. SQL FTS has faster indexing and is searches are very > fast. Lucene is optimized for returning small number of rows (as is SQL > FTS). Lucene does trimming which allows you to filter on the full-text index > by date (for example). > > Katmai (SQL 2008) should close this gap. > > -- > RelevantNoise.com - dedicated to mining blogs for business intelligence. > > Looking for a SQL Server replication book? > http://www.nwsu.com/0974973602.html > > Looking for a FAQ on Indexing Services/SQL FTS > http://www.indexserverfaq.com > "Vijay" <Vijay@discussions.microsoft.com> wrote in message > news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... > > Hello, > > > > We are using SQL Server 2000 and we are doing Full-Text Indexing on a > > table > > that has 5 Text Columns (we have Full Text Indexing on these 5 columns). > > There are around 1.5 million records in this table. And the size of the > > Full-Text Catalog is around 1 GB. > > > > The performance issue we are facing is some searches are slow. If I search > > for "to kill a mockingbird" it returns results very quickly and if I > > search > > for "numbers" the search is slower. So we are looking to increase the > > search > > speed. And also we need to get paginated results in our application. > > > > So what options I have to try? > > > > I am thinking of these options: > > . Upgrade to SQL Server 2005 and partition this table based on the year. > > Is > > it possible to create different Full-Text Catalogs for each partition? > > . I just learnt about Lucene4DB. > > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . Has > > anyone used this tool in their application? Will there be a gain in > > performance in the search speed and maintaining the indexes. > > > > Thanks > > Vijay > > > >
SQL 2005 is a great choice.
Lucene is also one. SQL FTS has faster indexing and is searches are very fast. Lucene is optimized for returning small number of rows (as is SQL FTS). Lucene does trimming which allows you to filter on the full-text index by date (for example). Katmai (SQL 2008) should close this gap. -- RelevantNoise.com - dedicated to mining blogs for business intelligence. Looking for a SQL Server replication book? http://www.nwsu.com/0974973602.html Looking for a FAQ on Indexing Services/SQL FTS http://www.indexserverfaq.com [quoted text, click to view] "Vijay" <Vijay@discussions.microsoft.com> wrote in message news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... > Hello, > > We are using SQL Server 2000 and we are doing Full-Text Indexing on a > table > that has 5 Text Columns (we have Full Text Indexing on these 5 columns). > There are around 1.5 million records in this table. And the size of the > Full-Text Catalog is around 1 GB. > > The performance issue we are facing is some searches are slow. If I search > for "to kill a mockingbird" it returns results very quickly and if I > search > for "numbers" the search is slower. So we are looking to increase the > search > speed. And also we need to get paginated results in our application. > > So what options I have to try? > > I am thinking of these options: > . Upgrade to SQL Server 2005 and partition this table based on the year. > Is > it possible to create different Full-Text Catalogs for each partition? > . I just learnt about Lucene4DB. > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . Has > anyone used this tool in their application? Will there be a gain in > performance in the search speed and maintaining the indexes. > > Thanks > Vijay >
Thanks Hilary
Vijay [quoted text, click to view] "Hilary Cotter" wrote:
> No, one catalog per table partition or unpartitioned. > > I have no stats for comparison's between Lucene4DB or SQL Server. > > Katmai is scheduled for release in early Feb 2008 (IIRC). > > -- > RelevantNoise.com - dedicated to mining blogs for business intelligence. > > Looking for a SQL Server replication book? > http://www.nwsu.com/0974973602.html > > Looking for a FAQ on Indexing Services/SQL FTS > http://www.indexserverfaq.com > "Vijay" <Vijay@discussions.microsoft.com> wrote in message > news:56C8AB41-5885-48E2-B9EC-8A8691DDB682@microsoft.com... > > Hi Hilary Cotter, > > > > Thanks for your quick reply. > > I have a few followup questions: > > Can we create a FT Catalog for each partition in a partitioned table in > > SQL > > 2005? > > (If the FT Catalog is smaller then hopefully the searches will be faster) > > > > Can you point me to any sources that show the comparision of perfomance in > > Seraches and Indexing between Lucene4DB and SQL Server? > > > > When will Katmai (SQL 2008) release? > > > > Thanks in advance, > > Vijay > > > > "Hilary Cotter" wrote: > > > >> SQL 2005 is a great choice. > >> > >> Lucene is also one. SQL FTS has faster indexing and is searches are very > >> fast. Lucene is optimized for returning small number of rows (as is SQL > >> FTS). Lucene does trimming which allows you to filter on the full-text > >> index > >> by date (for example). > >> > >> Katmai (SQL 2008) should close this gap. > >> > >> -- > >> RelevantNoise.com - dedicated to mining blogs for business intelligence. > >> > >> Looking for a SQL Server replication book? > >> http://www.nwsu.com/0974973602.html > >> > >> Looking for a FAQ on Indexing Services/SQL FTS > >> http://www.indexserverfaq.com > >> "Vijay" <Vijay@discussions.microsoft.com> wrote in message > >> news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... > >> > Hello, > >> > > >> > We are using SQL Server 2000 and we are doing Full-Text Indexing on a > >> > table > >> > that has 5 Text Columns (we have Full Text Indexing on these 5 > >> > columns). > >> > There are around 1.5 million records in this table. And the size of the > >> > Full-Text Catalog is around 1 GB. > >> > > >> > The performance issue we are facing is some searches are slow. If I > >> > search > >> > for "to kill a mockingbird" it returns results very quickly and if I > >> > search > >> > for "numbers" the search is slower. So we are looking to increase the > >> > search > >> > speed. And also we need to get paginated results in our application. > >> > > >> > So what options I have to try? > >> > > >> > I am thinking of these options: > >> > . Upgrade to SQL Server 2005 and partition this table based on the > >> > year. > >> > Is > >> > it possible to create different Full-Text Catalogs for each partition? > >> > . I just learnt about Lucene4DB. > >> > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . > >> > Has > >> > anyone used this tool in their application? Will there be a gain in > >> > performance in the search speed and maintaining the indexes. > >> > > >> > Thanks > >> > Vijay > >> > > >> > >> > >> > >
No, one catalog per table partition or unpartitioned.
I have no stats for comparison's between Lucene4DB or SQL Server. Katmai is scheduled for release in early Feb 2008 (IIRC). -- RelevantNoise.com - dedicated to mining blogs for business intelligence. Looking for a SQL Server replication book? http://www.nwsu.com/0974973602.html Looking for a FAQ on Indexing Services/SQL FTS http://www.indexserverfaq.com [quoted text, click to view] "Vijay" <Vijay@discussions.microsoft.com> wrote in message news:56C8AB41-5885-48E2-B9EC-8A8691DDB682@microsoft.com... > Hi Hilary Cotter, > > Thanks for your quick reply. > I have a few followup questions: > Can we create a FT Catalog for each partition in a partitioned table in > SQL > 2005? > (If the FT Catalog is smaller then hopefully the searches will be faster) > > Can you point me to any sources that show the comparision of perfomance in > Seraches and Indexing between Lucene4DB and SQL Server? > > When will Katmai (SQL 2008) release? > > Thanks in advance, > Vijay > > "Hilary Cotter" wrote: > >> SQL 2005 is a great choice. >> >> Lucene is also one. SQL FTS has faster indexing and is searches are very >> fast. Lucene is optimized for returning small number of rows (as is SQL >> FTS). Lucene does trimming which allows you to filter on the full-text >> index >> by date (for example). >> >> Katmai (SQL 2008) should close this gap. >> >> -- >> RelevantNoise.com - dedicated to mining blogs for business intelligence. >> >> Looking for a SQL Server replication book? >> http://www.nwsu.com/0974973602.html >> >> Looking for a FAQ on Indexing Services/SQL FTS >> http://www.indexserverfaq.com >> "Vijay" <Vijay@discussions.microsoft.com> wrote in message >> news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... >> > Hello, >> > >> > We are using SQL Server 2000 and we are doing Full-Text Indexing on a >> > table >> > that has 5 Text Columns (we have Full Text Indexing on these 5 >> > columns). >> > There are around 1.5 million records in this table. And the size of the >> > Full-Text Catalog is around 1 GB. >> > >> > The performance issue we are facing is some searches are slow. If I >> > search >> > for "to kill a mockingbird" it returns results very quickly and if I >> > search >> > for "numbers" the search is slower. So we are looking to increase the >> > search >> > speed. And also we need to get paginated results in our application. >> > >> > So what options I have to try? >> > >> > I am thinking of these options: >> > . Upgrade to SQL Server 2005 and partition this table based on the >> > year. >> > Is >> > it possible to create different Full-Text Catalogs for each partition? >> > . I just learnt about Lucene4DB. >> > http://www.netomatix.com/products/documentmanagement/lucene4db.aspx . >> > Has >> > anyone used this tool in their application? Will there be a gain in >> > performance in the search speed and maintaining the indexes. >> > >> > Thanks >> > Vijay >> > >> >> >>
Hello Hilary,
Katmai is being launched in Feb 2008 but won't go to RTM until later in the year. Simon Sabin SQL Server MVP http://sqlblogcasts.com/blogs/simons [quoted text, click to view] > No, one catalog per table partition or unpartitioned. > > I have no stats for comparison's between Lucene4DB or SQL Server. > > Katmai is scheduled for release in early Feb 2008 (IIRC). > > Looking for a SQL Server replication book? > http://www.nwsu.com/0974973602.html > Looking for a FAQ on Indexing Services/SQL FTS > http://www.indexserverfaq.com > "Vijay" <Vijay@discussions.microsoft.com> wrote in message > news:56C8AB41-5885-48E2-B9EC-8A8691DDB682@microsoft.com... >> Hi Hilary Cotter, >> >> Thanks for your quick reply. >> I have a few followup questions: >> Can we create a FT Catalog for each partition in a partitioned table >> in >> SQL >> 2005? >> (If the FT Catalog is smaller then hopefully the searches will be >> faster) >> Can you point me to any sources that show the comparision of >> perfomance in Seraches and Indexing between Lucene4DB and SQL Server? >> >> When will Katmai (SQL 2008) release? >> >> Thanks in advance, >> Vijay >> "Hilary Cotter" wrote: >> >>> SQL 2005 is a great choice. >>> >>> Lucene is also one. SQL FTS has faster indexing and is searches are >>> very >>> fast. Lucene is optimized for returning small number of rows (as is >>> SQL >>> FTS). Lucene does trimming which allows you to filter on the >>> full-text >>> index >>> by date (for example). >>> Katmai (SQL 2008) should close this gap. >>> >>> -- RelevantNoise.com - dedicated to mining blogs for business >>> intelligence. >>> >>> Looking for a SQL Server replication book? >>> http://www.nwsu.com/0974973602.html >>> Looking for a FAQ on Indexing Services/SQL FTS >>> http://www.indexserverfaq.com >>> "Vijay" <Vijay@discussions.microsoft.com> wrote in message >>> news:F483AB25-FB29-48E7-9CCB-09E298718F0A@microsoft.com... >>>> Hello, >>>> >>>> We are using SQL Server 2000 and we are doing Full-Text Indexing on >>>> a >>>> table >>>> that has 5 Text Columns (we have Full Text Indexing on these 5 >>>> columns). >>>> There are around 1.5 million records in this table. And the size of >>>> the >>>> Full-Text Catalog is around 1 GB. >>>> The performance issue we are facing is some searches are slow. If I >>>> search >>>> for "to kill a mockingbird" it returns results very quickly and if >>>> I >>>> search >>>> for "numbers" the search is slower. So we are looking to increase >>>> the >>>> search >>>> speed. And also we need to get paginated results in our >>>> application. >>>> So what options I have to try? >>>> >>>> I am thinking of these options: >>>> . Upgrade to SQL Server 2005 and partition this table based on the >>>> year. >>>> Is >>>> it possible to create different Full-Text Catalogs for each >>>> partition? >>>> . I just learnt about Lucene4DB. >>>> http://www.netomatix.com/products/documentmanagement/lucene4db.aspx >>>> . >>>> Has >>>> anyone used this tool in their application? Will there be a gain in >>>> performance in the search speed and maintaining the indexes. >>>> Thanks >>>> Vijay
Don't see what you're looking for? Try a search.
|
June 2003
July 2003
August 2003
September 2003
October 2003
November 2003
December 2003
January 2004
February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
| home · blog · groups | Questions? Comments? Contact the dn |