Groups | Blog | Home
all groups > sql server replication > october 2006 >

sql server replication : Subscriber Transaction Logs Too Large


Lucas Davenport
10/11/2006 8:16:02 AM
Hello everyone,

I am having a problem with transaction logs on my subscriber server in a
snapshot/transactional replication environment. My publisher (server A) has
six (6) databases available for subscription. My subscriber (server B) pulls
it's subscription from server A for reporting purposes every two hours. The
log files (db.ldf) and the transaction log backup files are enormous. I've
attempted truncating the logs, but they won't budge. I have also tried
skrinking to files, but that doesn't work either. This is my first venture
into replication, and I finally got it to work correctly, only to find out
that my reporting server stopped this morning because it was out of space.

For now, I dropped the databases and restored them from a backup (forcing
the server to recreate new files for the MDF and LDF). This works, but I do
not want to do this every two weeks. Any suggestion on how I can get the log
files truncated on a regualr basis or generally reduced in size overall?

Lucas Davenport
10/11/2006 9:09:02 AM
I presume this could be done from a script and scheduled with the SQL Server
Agent to occur at the beginning and end of the snapshot replication. For
instance, set the database to bulk-logged before replication - let the
replication occur - set the database back to full-recovery model. Is that
right?

If so, I think I can handle that part. The part I have a question about is
dropping or dumping the log every 15 minutes. I'm a bit of a novice at this
stuff, so would you mind elaborating a bit on that portion of your suggestion?

Lastly, I think I can handle shrinking the log file 2 or more times. I can
do that from a scheduled task at the completion of the replication event and
just add it to the above series of events, right?

Thanks a bunch for your assistance.

[quoted text, click to view]
Lucas Davenport
10/11/2006 11:26:02 AM
Thanks for all your help - I really appreciate it. In your last reply, you
mentioned not shrinking the file too much on SQL 2000. I forgot to mention in
my original post tha I am doing replication between SQL 2000 (Server A) and
SQL 2005 (Server B). That doesn't change anything you mentioned, does it?

Again, thanks for all your help.

[quoted text, click to view]
Lucas Davenport
10/11/2006 11:27:03 AM
Thanks for all your help - I really appreciate it. In your last reply, you
mentioned not shrinking the file too much on SQL 2000. I forgot to mention in
my original post tha I am doing replication between SQL 2000 (Server A) and
SQL 2005 (Server B). That doesn't change anything you mentioned, does it?

Again, thanks for all your help.


[quoted text, click to view]
Lucas Davenport
10/11/2006 11:27:03 AM
Thanks for all your help - I really appreciate it. In your last reply, you
mentioned not shrinking the file too much on SQL 2000. I forgot to mention in
my original post tha I am doing replication between SQL 2000 (Server A) and
SQL 2005 (Server B). That doesn't change anything you mentioned, does it?

Again, thanks for all your help.

[quoted text, click to view]
Hilary Cotter
10/11/2006 11:27:09 AM
Hello Lucas. I would advise you to put the subscriber database into bulk
logged recovery model when the snapshot is being applied. After that put it
in full recovery model and dump the log every 15 minutes or so.

You will need to shrink the log 2-4 or possibly more times to get it to a
manageable size after putting it in full recover model.

--
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.

This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com



"Lucas Davenport" <Lucas Davenport@discussions.microsoft.com> wrote in
message news:DD8789BC-EE71-4B71-8C24-0489ABC004ED@microsoft.com...
[quoted text, click to view]

Hilary Cotter
10/11/2006 1:20:47 PM
the command to dump the log is

Backup log DatabaseName to disk='test.bak'

TO change the recovery model you can do an
alter database databaseName
set RECOVERY FULL

alter database databaseName
set BULK_LOGGED

You don't want to shrink the log too much in SQL 2000. Shrink it to the
point where it doesn't autogrow between dumps.
--
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.

This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com



"Lucas Davenport" <LucasDavenport@discussions.microsoft.com> wrote in
message news:14B7EE07-0074-461F-BB04-6851E9A91C0B@microsoft.com...
[quoted text, click to view]

Hilary Cotter
10/11/2006 2:42:25 PM
Its not so critical to shrink the db files or transaction logs in SQL 2005.
But other than that everything else applies.

--
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.

This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com



"Lucas Davenport" <LucasDavenport@discussions.microsoft.com> wrote in
message news:7AF9D08B-4DD1-4E7B-89C3-84CC9298DAA0@microsoft.com...
[quoted text, click to view]

Paul Ibison
10/11/2006 5:26:28 PM
Lucas,
if you have tried truncating the log and there's no change in space used
then there's definitely an issue and running DBCC OPENTRAN will let you know
if there is an open transaction which is blocking the truncate. If there is,
then run DBCC INPUTBUFFER to see what it is doing and make a decision
whether to remove it (KILL) or not. After that, i If you aren't bothered in
point-in-time recovery then I'd advise Simple Recovery Mode to make your
life easy. If you need point in time recovery, schedule a frequent log
backup and this should be enough.
Cheers,
Paul Ibison SQL Server MVP, www.replicationanswers.com .

zerg2k NO[at]SPAM yahoo.com
10/13/2006 2:45:00 PM
Well, I for one am very happy not to be alone in this issue. I do get
exactly your problem and For Paul and Hilary this is how it goes.
Bulk-Logged DOES NOT helps, Log backup DONT HELP either. I get this
issue when the initial snapshot is being applied and apparently the
server cant do anything about it only after the distribution agent is
done with bcp. If the file happens to be large ( like it does at random
times, than you MS for concurrent snapshot unevenly distributed bcp
files) no truncation is happening and a massive log size explosion
comes along. I have been working with MS India to no avail ( they don't
want to admit the bug because is difficult to reproduce grrrr).

Hilary or Paul with your influence in MS due to your MVP condition, I
would really appreciate if you could dig deeper this issue for us.

Kind Regards,

Noel
Sr. DBA




[quoted text, click to view]
zerg2k NO[at]SPAM yahoo.com
10/14/2006 8:47:51 PM

[quoted text, click to view]

Paul,
Thanks, for the reply. Yes I have followed the thread I started ;-),
lots of good stuff there (Thanks Raymond) and I will try all the
recomendations as soon as I go back to work. I however think the main
problem lies with Transaction log Bulk operations performed through
bcp. Unfortunately, Raymond stands at snapshot generation point in
time, which I think is a very good step forward but definitely there is
more to it than that... we'll see.

Regards,
Noel
Paul Ibison
10/14/2006 8:56:31 PM
Hi Noel - the MVP award does give some (limited) access to the relevant
product group, so I'll raise it there and keep you posted.
Rgds,
Paul Ibison

Paul Ibison
10/14/2006 9:32:27 PM
Actually Noel - have you been checking the thread "Re: transaction log too
large when applying snapshot ?".
Raymond Mak has answered this in more detail than I've seen anywhere before
along with some undocumented workarounds.
Cheers,
Paul Ibison SQL Server MVP, www.replicationanswers.com .

Paul Ibison
10/16/2006 10:29:04 AM
Sorry Noel - didn't realise you were the same guy. Please let us know how
raymond's recommendations go.
Cheers,
Paul Ibison SQL Server MVP, www.replicationanswers.com .

AddThis Social Bookmark Button