Showing posts with label scenario. Show all posts
Showing posts with label scenario. Show all posts

Thursday, March 29, 2012

Alternate Synchronization Server Schema Change Problem

Greetings:
We have set up a test scenario in Sql Server 2000 using merge replication
with:
A primary synchronization server (Publisher A)
An alternate synchronization partner (Publisher B)
Two subscribers to both publishers (Subscriber Y and Subscriber Z)
After some initial difficulties, we were able to get the alternate synch
partner running where both subscribers will successfully synch to either
machine without difficulty. (See thread “Alternate Synchronization Partner
-- Error”, originally posted on March 10.)
The next part of testing phase before going live is to test how schema
changes occur with alternate synch partners. We find NO documentation on
this subject anywhere, so if anyone has resources on how to do schema changes
where an alternate synchronization partner is involved, we would be very,
very grateful.
Our basic question: How are schema changes done where an alternate
synchronization partner is involved? We want to know exact procedures to do
the following:
-- Add a column
-- Drop a column
-- Add a table
We have tried to add a column as follows:
1.Used “sp_repladdcolumn” and added a column to one table in Publisher A,
the primary publisher. (See next paragraph on warning received.)
2.Synchronized the Publisher B, the alternate synch partner and it brought
in the column.
3.Synchronized Subscriber Y to Publisher A, and it brought in the column to
the subscriber.
At #1, when “sp_repladdcolumn” ran, I received the following warning:
“Cannot add rows to sysdepends for the current stored procedure because it
depends on the missing object 'sp_sel_D536588A75244907040F5997233E41C8_pal'.
The stored procedure will still be created.”
The Microsoft knowledge base article 811483 says this warning can be
ignored, but I wonder about being able to ignore this warning because of the
results we received described below.
Results from above process:
-- NO DATA UPDATES FROM PUBLISHER A WILL PROPAGATE TO EITHER PUBLISHER B,
THE ALTERNATE, OR TO SUBSCRIBER Y WHEN SYNCRHONIZING. New records added at
Publisher A do propagate, but no updates of existing data will. Inserts
propagate – updates do not propagate.
-- In addition, all data changes, inserts, and deletes made at either
Publisher B, the alternate synch partner, or at Subscriber Y, DO propagate
back to the primary Publisher A.
When I found that no data updates would propagate to the alternate synch
partner or the subscribers, I then
1.Used “sp_repldropcolumn” to drop the column I had added on Publisher A,
the primary.
2.Synchronized Publisher B, the alternate synch partner and it dropped the
column.
3.Synchronized Subscriber Y and it dropped the column.
I then tested data updates from Publisher A, the primary, and the updates
all began to propagate to Publisher B, the alternate and Subscriber Y once
again.
Summary:
-- A column was added to the Publisher A and the schema change propagated
to Publisher B and Subscriber Y.
-- After the schema change was propagated, any data UPDATES on Publisher A
would not propagate to the Publisher B or Subscriber Y. INSERTS however,
would propagate.
-- The column was dropped at Publisher A and that schema change propagated
to Publisher B and Subscriber Y.
-- Data updates from Publish A began to propagate again go Publisher B and
Subscriber Y.
What is wrong with the procedures I am using to make schema changes that are
causing the primary synchronization server to stop propagating data updates
after the column is added, but when the column is dropped, updates begin to
propagate again?
Any procedures for schema changes when an alternate synchronization partner
is involved would be very much appreciated.
Thank you.
Bill
Hi Bill,
From your descriptions, I understood that you would like to why
sp_repladdcolumn does not take effect. If I have misunderstood your
concern, please feel free to point it out.
First of all, please understand that replication issues tend to be very
complex and hard to troubleshoot in newsgroups. If you need detail and
prompt assistance, I recommend that you open a Support incident with
Microsoft Customer Service and Support (CSS) so that a dedicated Support
Professional can work with you in a more timely and efficient manner. If
you need any help in this regard, please let me know.
For a complete list of Microsoft Customer Service and Support phone
numbers, please go to the following address on the World Wide Web:
<http://support.microsoft.com/directory/overview.asp>
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
Secondly, based on my knowledge, once the column is added at the original
publisher, the merge agent between the original publisher and the alternate
publisher has to be run so that the new schema is sent to the alternate
publisher. Also, the merge agent between the original publisher and the
subscriber needs to be run so that the schema definition is sent here. As
long as the schema of the replicating pairs match, there are no issues.
It seems your process is OK and it's weird that you are not able to
synchronize the data. Do you find any other error messages in Event Logs or
SQL Logs?
You may generate the file as following Knowledge Base article asked and
then send to me direcly v-mingqc@.online.microsoft.com (plase ensure to
remove 'online' in the email address as it's only for SPAM)
HOW TO: Enable Replication Agents for Logging to Output Files in SQL Server
http://support.microsoft.com/?id=312292
Thank you for your patience and corporation. If you have any questions or
concerns, don't hesitate to let me know. We are always here to be of
assistance!
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
================================================== ===
This posting is provided "AS IS" with no warranties, and confers no rights.
|||Michael:
Thanks for the information. I was preparing to send you the logs when I
suddenly wondered if I needed to reinitialize the pull subscriptions. So I
decided to try that first, and it worked. After reinitializing all
subscriptions, all data changes began to synchronize with no errors or
problems.
I then ran several additional tests with adding and dropping new columns.
When I reinitialized the subscriptions, everything synchronized. When I did
not reinitialize, the schema would propagate, but data changes would not.
This is clearly different from when there is no alternate synch server. We
have added and deleted columns many times before with sp_repladdcolumn or
sp_repldropcolumn when no alternate synch server was involved. Data changes
always continued to synchronize properly in that circumstance. Perhaps some
note needs to be put in BOL about this difference when an alternate
synchronization server is involved? Just a suggestion that might save others
time.
The next test I am going to try is to add a new table (article) to our test
database and see if I can get it to propagate to the alternate synch server
and the two subscribers we have.
You have been a great help in pointing me the right direction. I really
appreciate it.
Bill
"Michael Cheng [MSFT]" wrote:

> Hi Bill,
> From your descriptions, I understood that you would like to why
> sp_repladdcolumn does not take effect. If I have misunderstood your
> concern, please feel free to point it out.
> First of all, please understand that replication issues tend to be very
> complex and hard to troubleshoot in newsgroups. If you need detail and
> prompt assistance, I recommend that you open a Support incident with
> Microsoft Customer Service and Support (CSS) so that a dedicated Support
> Professional can work with you in a more timely and efficient manner. If
> you need any help in this regard, please let me know.
> For a complete list of Microsoft Customer Service and Support phone
> numbers, please go to the following address on the World Wide Web:
> <http://support.microsoft.com/directory/overview.asp>
> If you are outside the US please see http://support.microsoft.com for
> regional support phone numbers.
> Secondly, based on my knowledge, once the column is added at the original
> publisher, the merge agent between the original publisher and the alternate
> publisher has to be run so that the new schema is sent to the alternate
> publisher. Also, the merge agent between the original publisher and the
> subscriber needs to be run so that the schema definition is sent here. As
> long as the schema of the replicating pairs match, there are no issues.
> It seems your process is OK and it's weird that you are not able to
> synchronize the data. Do you find any other error messages in Event Logs or
> SQL Logs?
> You may generate the file as following Knowledge Base article asked and
> then send to me direcly v-mingqc@.online.microsoft.com (plase ensure to
> remove 'online' in the email address as it's only for SPAM)
> HOW TO: Enable Replication Agents for Logging to Output Files in SQL Server
> http://support.microsoft.com/?id=312292
> Thank you for your patience and corporation. If you have any questions or
> concerns, don't hesitate to let me know. We are always here to be of
> assistance!
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ================================================== ===
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
|||Hi Bill,
I searched internal but limited resource on this topic about why
reinitializing make all things go smoothly as I cannot reproduce it on my
side as you described. If you want to fingure out the root cause of this
issue, you'd better send the logs to our PSS guys as Replication issues
might turn out to be very complicated.
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
================================================== ===
This posting is provided "AS IS" with no warranties, and confers no rights.
|||Hi Michael;
With the reinitialization of the subscriptions, the schema changes and data
changes synchronized perfectly on numerous tests. I put together a process
for us to use in these circumstances because it is a bit different between
add/drop a column and adding a new table. In any case, I did get them work
fine with the altternate synch server.
If I did not reinitialize the subscriptions, the schema would propagate, but
the data changes would not from the primary server. I tried this in several
tests and could not get it go, but since I have a procedure that works now,
my business partners and I are happy with it and will go live in the next
month or two with our alternate synchronization partner.
I appreciate all your help and suggestions.
Bill
"Michael Cheng [MSFT]" wrote:

> Hi Bill,
> I searched internal but limited resource on this topic about why
> reinitializing make all things go smoothly as I cannot reproduce it on my
> side as you described. If you want to fingure out the root cause of this
> issue, you'd better send the logs to our PSS guys as Replication issues
> might turn out to be very complicated.
>
> Sincerely yours,
> Michael Cheng
> Microsoft Online Partner Support
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ================================================== ===
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
|||Hi Bill,
Thanks for your perfect summary and I believe others will also benefit from
your great work
Sincerely yours,
Michael Cheng
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
================================================== ===
This posting is provided "AS IS" with no warranties, and confers no rights.

Thursday, March 8, 2012

ALTER PARTITION FUNCTION and I/O

We have a sliding window scenario where every day we add a new day and trim
off an old day from our partition function and scheme.
I read somewhere that in this scenario it is better to keep Partition1 empty
and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that you
will incurr additional I/O if you do it this way.
Before I test this, can anyone confirm this. We have 30 million row
partitions and can't afford any I/O while we MERGE Partition1. I am of the
opinion that provided Partition1 is empty, the MERGE will always be a
metadata operation only.
-- switch out Partition1 then MERGE
boundary_id row_count
1 30000000
2 30000000
3 30000000
4 30000000
5 30000000
6 30000000
7 30000000
-- or switch out partition2 then MERGE
boundary_id row_count
1 0
2 30000000
3 30000000
4 30000000
5 30000000
6 30000000
7 30000000
which is less I/O? I dont want to be moving data around on disk.
-- cranfield, DBA
> I read somewhere that in this scenario it is better to keep Partition1
> empty
> and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
> SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that
> you
> will incurr additional I/O if you do it this way.
The partition that includes the specified boundary must be empty in order to
avoid data movement during MERGE.

> -- switch out Partition1 then MERGE
> boundary_id row_count
> 1 30000000
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
No data movement will be needed if the function is RANGE LEFT (inclusive).
Data will need to be moved if RANGE RIGHT because the first boundary
includes data in the second partition.

> -- or switch out partition2 then MERGE
> boundary_id row_count
> 1 0
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
No data movement will be needed because both partitions 1 and 2 will be
empty during the MERGE. No data movement is needed in this case regardless
of LEFT or RIGHT.
Regarding general sliding window approaches, one usually wants to keep all
values for a given date in the same partition. The 2 basic techniques to
accomplish an efficient datetime based sliding window:
Method 1:
Specify RANGE LEFT with boundary values that includes the highest possible
datetime value (e.g. '2008-02-29T23:59:59.997'). To remove old data, switch
out partition 1 and MERGE the first partition.
Method 2:
Specify RANGE RIGHT with boundary values of the next date (e.g.
'2008-03-01T00:00:00'). To remove old data, switch out partition 2 and
MERGE the first partition. Partition 1 is always empty with this approach.
Hope this helps.
Dan Guzman
SQL Server MVP
"Cranfield" <alan_cranfield@.msn.co.za> wrote in message
news:A75BDFAA-9492-455A-92BE-12F4FFBCCED9@.microsoft.com...
> We have a sliding window scenario where every day we add a new day and
> trim
> off an old day from our partition function and scheme.
> I read somewhere that in this scenario it is better to keep Partition1
> empty
> and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
> SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that
> you
> will incurr additional I/O if you do it this way.
> Before I test this, can anyone confirm this. We have 30 million row
> partitions and can't afford any I/O while we MERGE Partition1. I am of the
> opinion that provided Partition1 is empty, the MERGE will always be a
> metadata operation only.
> -- switch out Partition1 then MERGE
> boundary_id row_count
> 1 30000000
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
> -- or switch out partition2 then MERGE
> boundary_id row_count
> 1 0
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
> which is less I/O? I dont want to be moving data around on disk.
> --
> -- cranfield, DBA
|||Hi Dan
Thankls for that. We've used RANGE RIGHT as we have the concept of a
Trading Day which is all data up to 21h30. Anything that comes in after that
goes to the next partition. In addition we have binary date representation as
our partition key. We will make sure we keep Partition1 empty and SWITCH OUT
Partition2 and then MERGE.
--e.g.
CREATE PARTITION FUNCTION [MyFunc](binary(16))
AS RANGE RIGHT FOR VALUES (
0x47966058000000000000000000000000,
0x4797B1D8000000000000000000000000,
0x47990358000000000000000000000000,
0x479A54D8000000000000000000000000,
0x479BA658000000000000000000000000,
0x479CF7D8000000000000000000000000,
0x479E4958000000000000000000000000,
0x479F9AD8000000000000000000000000,
0x47A0EC58000000000000000000000000,
0x47A23DD8000000000000000000000000,
0x47A38F58000000000000000000000000,
0x47A4E0D8000000000000000000000000,
0x47A63258000000000000000000000000,
0x47A783D8000000000000000000000000,
0x47A8D558000000000000000000000000,
0x47AA26D8000000000000000000000000,
0x47AB7858000000000000000000000000
)
-- equates to:
2008-01-22 21:30:00.000
2008-01-23 21:30:00.000
2008-01-24 21:30:00.000
2008-01-25 21:30:00.000
2008-01-26 21:30:00.000
2008-01-27 21:30:00.000
2008-01-28 21:30:00.000
2008-01-29 21:30:00.000
2008-01-30 21:30:00.000
2008-01-31 21:30:00.000
2008-02-01 21:30:00.000
2008-02-02 21:30:00.000
2008-02-03 21:30:00.000
2008-02-04 21:30:00.000
2008-02-05 21:30:00.000
-- cranfield, DBA
"Dan Guzman" wrote:

> The partition that includes the specified boundary must be empty in order to
> avoid data movement during MERGE.
>
> No data movement will be needed if the function is RANGE LEFT (inclusive).
> Data will need to be moved if RANGE RIGHT because the first boundary
> includes data in the second partition.
>
> No data movement will be needed because both partitions 1 and 2 will be
> empty during the MERGE. No data movement is needed in this case regardless
> of LEFT or RIGHT.
> Regarding general sliding window approaches, one usually wants to keep all
> values for a given date in the same partition. The 2 basic techniques to
> accomplish an efficient datetime based sliding window:
> Method 1:
> Specify RANGE LEFT with boundary values that includes the highest possible
> datetime value (e.g. '2008-02-29T23:59:59.997'). To remove old data, switch
> out partition 1 and MERGE the first partition.
> Method 2:
> Specify RANGE RIGHT with boundary values of the next date (e.g.
> '2008-03-01T00:00:00'). To remove old data, switch out partition 2 and
> MERGE the first partition. Partition 1 is always empty with this approach.
> --
> Hope this helps.
> Dan Guzman
> SQL Server MVP
> "Cranfield" <alan_cranfield@.msn.co.za> wrote in message
> news:A75BDFAA-9492-455A-92BE-12F4FFBCCED9@.microsoft.com...
>
|||> We will make sure we keep Partition1 empty and SWITCH OUT
> Partition2 and then MERGE.
It looks like you are in good shape then. I'm glad I was able to help out
out.
Dan Guzman
SQL Server MVP
"Cranfield" <alan_cranfield@.msn.co.za> wrote in message
news:34841B67-89A1-4AF0-8069-EE03F709B923@.microsoft.com...[vbcol=seagreen]
> Hi Dan
> Thankls for that. We've used RANGE RIGHT as we have the concept of a
> Trading Day which is all data up to 21h30. Anything that comes in after
> that
> goes to the next partition. In addition we have binary date representation
> as
> our partition key. We will make sure we keep Partition1 empty and SWITCH
> OUT
> Partition2 and then MERGE.
> --e.g.
> CREATE PARTITION FUNCTION [MyFunc](binary(16))
> AS RANGE RIGHT FOR VALUES (
> 0x47966058000000000000000000000000,
> 0x4797B1D8000000000000000000000000,
> 0x47990358000000000000000000000000,
> 0x479A54D8000000000000000000000000,
> 0x479BA658000000000000000000000000,
> 0x479CF7D8000000000000000000000000,
> 0x479E4958000000000000000000000000,
> 0x479F9AD8000000000000000000000000,
> 0x47A0EC58000000000000000000000000,
> 0x47A23DD8000000000000000000000000,
> 0x47A38F58000000000000000000000000,
> 0x47A4E0D8000000000000000000000000,
> 0x47A63258000000000000000000000000,
> 0x47A783D8000000000000000000000000,
> 0x47A8D558000000000000000000000000,
> 0x47AA26D8000000000000000000000000,
> 0x47AB7858000000000000000000000000
> )
>
> -- equates to:
> 2008-01-22 21:30:00.000
> 2008-01-23 21:30:00.000
> 2008-01-24 21:30:00.000
> 2008-01-25 21:30:00.000
> 2008-01-26 21:30:00.000
> 2008-01-27 21:30:00.000
> 2008-01-28 21:30:00.000
> 2008-01-29 21:30:00.000
> 2008-01-30 21:30:00.000
> 2008-01-31 21:30:00.000
> 2008-02-01 21:30:00.000
> 2008-02-02 21:30:00.000
> 2008-02-03 21:30:00.000
> 2008-02-04 21:30:00.000
> 2008-02-05 21:30:00.000
>
> --
> -- cranfield, DBA
>
> "Dan Guzman" wrote:

ALTER PARTITION FUNCTION and I/O

We have a sliding window scenario where every day we add a new day and trim
off an old day from our partition function and scheme.
I read somewhere that in this scenario it is better to keep Partition1 empty
and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that you
will incurr additional I/O if you do it this way.
Before I test this, can anyone confirm this. We have 30 million row
partitions and can't afford any I/O while we MERGE Partition1. I am of the
opinion that provided Partition1 is empty, the MERGE will always be a
metadata operation only.
-- switch out Partition1 then MERGE
boundary_id row_count
1 30000000
2 30000000
3 30000000
4 30000000
5 30000000
6 30000000
7 30000000
-- or switch out partition2 then MERGE
boundary_id row_count
1 0
2 30000000
3 30000000
4 30000000
5 30000000
6 30000000
7 30000000
which is less I/O? I dont want to be moving data around on disk.
--
-- cranfield, DBA> I read somewhere that in this scenario it is better to keep Partition1
> empty
> and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
> SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that
> you
> will incurr additional I/O if you do it this way.
The partition that includes the specified boundary must be empty in order to
avoid data movement during MERGE.
> -- switch out Partition1 then MERGE
> boundary_id row_count
> 1 30000000
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
No data movement will be needed if the function is RANGE LEFT (inclusive).
Data will need to be moved if RANGE RIGHT because the first boundary
includes data in the second partition.
> -- or switch out partition2 then MERGE
> boundary_id row_count
> 1 0
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
No data movement will be needed because both partitions 1 and 2 will be
empty during the MERGE. No data movement is needed in this case regardless
of LEFT or RIGHT.
Regarding general sliding window approaches, one usually wants to keep all
values for a given date in the same partition. The 2 basic techniques to
accomplish an efficient datetime based sliding window:
Method 1:
Specify RANGE LEFT with boundary values that includes the highest possible
datetime value (e.g. '2008-02-29T23:59:59.997'). To remove old data, switch
out partition 1 and MERGE the first partition.
Method 2:
Specify RANGE RIGHT with boundary values of the next date (e.g.
'2008-03-01T00:00:00'). To remove old data, switch out partition 2 and
MERGE the first partition. Partition 1 is always empty with this approach.
--
Hope this helps.
Dan Guzman
SQL Server MVP
"Cranfield" <alan_cranfield@.msn.co.za> wrote in message
news:A75BDFAA-9492-455A-92BE-12F4FFBCCED9@.microsoft.com...
> We have a sliding window scenario where every day we add a new day and
> trim
> off an old day from our partition function and scheme.
> I read somewhere that in this scenario it is better to keep Partition1
> empty
> and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
> SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that
> you
> will incurr additional I/O if you do it this way.
> Before I test this, can anyone confirm this. We have 30 million row
> partitions and can't afford any I/O while we MERGE Partition1. I am of the
> opinion that provided Partition1 is empty, the MERGE will always be a
> metadata operation only.
> -- switch out Partition1 then MERGE
> boundary_id row_count
> 1 30000000
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
> -- or switch out partition2 then MERGE
> boundary_id row_count
> 1 0
> 2 30000000
> 3 30000000
> 4 30000000
> 5 30000000
> 6 30000000
> 7 30000000
> which is less I/O? I dont want to be moving data around on disk.
> --
> -- cranfield, DBA|||Hi Dan
Thankls for that. We've used RANGE RIGHT as we have the concept of a
Trading Day which is all data up to 21h30. Anything that comes in after that
goes to the next partition. In addition we have binary date representation as
our partition key. We will make sure we keep Partition1 empty and SWITCH OUT
Partition2 and then MERGE.
--e.g.
CREATE PARTITION FUNCTION [MyFunc](binary(16))
AS RANGE RIGHT FOR VALUES (
0x47966058000000000000000000000000,
0x4797B1D8000000000000000000000000,
0x47990358000000000000000000000000,
0x479A54D8000000000000000000000000,
0x479BA658000000000000000000000000,
0x479CF7D8000000000000000000000000,
0x479E4958000000000000000000000000,
0x479F9AD8000000000000000000000000,
0x47A0EC58000000000000000000000000,
0x47A23DD8000000000000000000000000,
0x47A38F58000000000000000000000000,
0x47A4E0D8000000000000000000000000,
0x47A63258000000000000000000000000,
0x47A783D8000000000000000000000000,
0x47A8D558000000000000000000000000,
0x47AA26D8000000000000000000000000,
0x47AB7858000000000000000000000000
)
-- equates to:
2008-01-22 21:30:00.000
2008-01-23 21:30:00.000
2008-01-24 21:30:00.000
2008-01-25 21:30:00.000
2008-01-26 21:30:00.000
2008-01-27 21:30:00.000
2008-01-28 21:30:00.000
2008-01-29 21:30:00.000
2008-01-30 21:30:00.000
2008-01-31 21:30:00.000
2008-02-01 21:30:00.000
2008-02-02 21:30:00.000
2008-02-03 21:30:00.000
2008-02-04 21:30:00.000
2008-02-05 21:30:00.000
-- cranfield, DBA
"Dan Guzman" wrote:
> > I read somewhere that in this scenario it is better to keep Partition1
> > empty
> > and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
> > SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that
> > you
> > will incurr additional I/O if you do it this way.
> The partition that includes the specified boundary must be empty in order to
> avoid data movement during MERGE.
> > -- switch out Partition1 then MERGE
> > boundary_id row_count
> > 1 30000000
> > 2 30000000
> > 3 30000000
> > 4 30000000
> > 5 30000000
> > 6 30000000
> > 7 30000000
> No data movement will be needed if the function is RANGE LEFT (inclusive).
> Data will need to be moved if RANGE RIGHT because the first boundary
> includes data in the second partition.
> > -- or switch out partition2 then MERGE
> > boundary_id row_count
> > 1 0
> > 2 30000000
> > 3 30000000
> > 4 30000000
> > 5 30000000
> > 6 30000000
> > 7 30000000
> No data movement will be needed because both partitions 1 and 2 will be
> empty during the MERGE. No data movement is needed in this case regardless
> of LEFT or RIGHT.
> Regarding general sliding window approaches, one usually wants to keep all
> values for a given date in the same partition. The 2 basic techniques to
> accomplish an efficient datetime based sliding window:
> Method 1:
> Specify RANGE LEFT with boundary values that includes the highest possible
> datetime value (e.g. '2008-02-29T23:59:59.997'). To remove old data, switch
> out partition 1 and MERGE the first partition.
> Method 2:
> Specify RANGE RIGHT with boundary values of the next date (e.g.
> '2008-03-01T00:00:00'). To remove old data, switch out partition 2 and
> MERGE the first partition. Partition 1 is always empty with this approach.
> --
> Hope this helps.
> Dan Guzman
> SQL Server MVP
> "Cranfield" <alan_cranfield@.msn.co.za> wrote in message
> news:A75BDFAA-9492-455A-92BE-12F4FFBCCED9@.microsoft.com...
> > We have a sliding window scenario where every day we add a new day and
> > trim
> > off an old day from our partition function and scheme.
> >
> > I read somewhere that in this scenario it is better to keep Partition1
> > empty
> > and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
> > SWITCHing out Partition1 and then MERGEing Partition 1. Reason being that
> > you
> > will incurr additional I/O if you do it this way.
> >
> > Before I test this, can anyone confirm this. We have 30 million row
> > partitions and can't afford any I/O while we MERGE Partition1. I am of the
> > opinion that provided Partition1 is empty, the MERGE will always be a
> > metadata operation only.
> >
> > -- switch out Partition1 then MERGE
> > boundary_id row_count
> > 1 30000000
> > 2 30000000
> > 3 30000000
> > 4 30000000
> > 5 30000000
> > 6 30000000
> > 7 30000000
> >
> > -- or switch out partition2 then MERGE
> > boundary_id row_count
> > 1 0
> > 2 30000000
> > 3 30000000
> > 4 30000000
> > 5 30000000
> > 6 30000000
> > 7 30000000
> >
> > which is less I/O? I dont want to be moving data around on disk.
> > --
> > -- cranfield, DBA
>|||> We will make sure we keep Partition1 empty and SWITCH OUT
> Partition2 and then MERGE.
It looks like you are in good shape then. I'm glad I was able to help out
out.
--
Dan Guzman
SQL Server MVP
"Cranfield" <alan_cranfield@.msn.co.za> wrote in message
news:34841B67-89A1-4AF0-8069-EE03F709B923@.microsoft.com...
> Hi Dan
> Thankls for that. We've used RANGE RIGHT as we have the concept of a
> Trading Day which is all data up to 21h30. Anything that comes in after
> that
> goes to the next partition. In addition we have binary date representation
> as
> our partition key. We will make sure we keep Partition1 empty and SWITCH
> OUT
> Partition2 and then MERGE.
> --e.g.
> CREATE PARTITION FUNCTION [MyFunc](binary(16))
> AS RANGE RIGHT FOR VALUES (
> 0x47966058000000000000000000000000,
> 0x4797B1D8000000000000000000000000,
> 0x47990358000000000000000000000000,
> 0x479A54D8000000000000000000000000,
> 0x479BA658000000000000000000000000,
> 0x479CF7D8000000000000000000000000,
> 0x479E4958000000000000000000000000,
> 0x479F9AD8000000000000000000000000,
> 0x47A0EC58000000000000000000000000,
> 0x47A23DD8000000000000000000000000,
> 0x47A38F58000000000000000000000000,
> 0x47A4E0D8000000000000000000000000,
> 0x47A63258000000000000000000000000,
> 0x47A783D8000000000000000000000000,
> 0x47A8D558000000000000000000000000,
> 0x47AA26D8000000000000000000000000,
> 0x47AB7858000000000000000000000000
> )
>
> -- equates to:
> 2008-01-22 21:30:00.000
> 2008-01-23 21:30:00.000
> 2008-01-24 21:30:00.000
> 2008-01-25 21:30:00.000
> 2008-01-26 21:30:00.000
> 2008-01-27 21:30:00.000
> 2008-01-28 21:30:00.000
> 2008-01-29 21:30:00.000
> 2008-01-30 21:30:00.000
> 2008-01-31 21:30:00.000
> 2008-02-01 21:30:00.000
> 2008-02-02 21:30:00.000
> 2008-02-03 21:30:00.000
> 2008-02-04 21:30:00.000
> 2008-02-05 21:30:00.000
>
> --
> -- cranfield, DBA
>
> "Dan Guzman" wrote:
>> > I read somewhere that in this scenario it is better to keep Partition1
>> > empty
>> > and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
>> > SWITCHing out Partition1 and then MERGEing Partition 1. Reason being
>> > that
>> > you
>> > will incurr additional I/O if you do it this way.
>> The partition that includes the specified boundary must be empty in order
>> to
>> avoid data movement during MERGE.
>> > -- switch out Partition1 then MERGE
>> > boundary_id row_count
>> > 1 30000000
>> > 2 30000000
>> > 3 30000000
>> > 4 30000000
>> > 5 30000000
>> > 6 30000000
>> > 7 30000000
>> No data movement will be needed if the function is RANGE LEFT
>> (inclusive).
>> Data will need to be moved if RANGE RIGHT because the first boundary
>> includes data in the second partition.
>> > -- or switch out partition2 then MERGE
>> > boundary_id row_count
>> > 1 0
>> > 2 30000000
>> > 3 30000000
>> > 4 30000000
>> > 5 30000000
>> > 6 30000000
>> > 7 30000000
>> No data movement will be needed because both partitions 1 and 2 will be
>> empty during the MERGE. No data movement is needed in this case
>> regardless
>> of LEFT or RIGHT.
>> Regarding general sliding window approaches, one usually wants to keep
>> all
>> values for a given date in the same partition. The 2 basic techniques to
>> accomplish an efficient datetime based sliding window:
>> Method 1:
>> Specify RANGE LEFT with boundary values that includes the highest
>> possible
>> datetime value (e.g. '2008-02-29T23:59:59.997'). To remove old data,
>> switch
>> out partition 1 and MERGE the first partition.
>> Method 2:
>> Specify RANGE RIGHT with boundary values of the next date (e.g.
>> '2008-03-01T00:00:00'). To remove old data, switch out partition 2 and
>> MERGE the first partition. Partition 1 is always empty with this
>> approach.
>> --
>> Hope this helps.
>> Dan Guzman
>> SQL Server MVP
>> "Cranfield" <alan_cranfield@.msn.co.za> wrote in message
>> news:A75BDFAA-9492-455A-92BE-12F4FFBCCED9@.microsoft.com...
>> > We have a sliding window scenario where every day we add a new day and
>> > trim
>> > off an old day from our partition function and scheme.
>> >
>> > I read somewhere that in this scenario it is better to keep Partition1
>> > empty
>> > and always SWITCH out Partition2 and then MERGE Partition 1. Instead of
>> > SWITCHing out Partition1 and then MERGEing Partition 1. Reason being
>> > that
>> > you
>> > will incurr additional I/O if you do it this way.
>> >
>> > Before I test this, can anyone confirm this. We have 30 million row
>> > partitions and can't afford any I/O while we MERGE Partition1. I am of
>> > the
>> > opinion that provided Partition1 is empty, the MERGE will always be a
>> > metadata operation only.
>> >
>> > -- switch out Partition1 then MERGE
>> > boundary_id row_count
>> > 1 30000000
>> > 2 30000000
>> > 3 30000000
>> > 4 30000000
>> > 5 30000000
>> > 6 30000000
>> > 7 30000000
>> >
>> > -- or switch out partition2 then MERGE
>> > boundary_id row_count
>> > 1 0
>> > 2 30000000
>> > 3 30000000
>> > 4 30000000
>> > 5 30000000
>> > 6 30000000
>> > 7 30000000
>> >
>> > which is less I/O? I dont want to be moving data around on disk.
>> > --
>> > -- cranfield, DBA

Alter existing table to add a IDENTITY column

Hi this is my first visit to the MSSQL forum with a question.

Let me explain the scenario,

I have a table say clients table with the structure like id,foo,etc.. and lots of records on it. But the issue is this id column is not an IDENTITY column.
But the values for the Id column dont repeat since it has handled from the application level.Now I need to change this id column as an IDENTITY column with out loosing the records on the table.

If any of you can guide me over this problem, its highly appreciated.
Thanks.Hi
you have 2 choice.

1) Alter table change the column propertyes to identity column but i don't remember if this option keep old id value. you must try.
2) Copy all data of table in another table.
Change the column property Generate an insert statement from the copied dato to new table . Before you execute the insert statement you mast write SET Identity insert ON for the destioantion table.
When you finished execute Set Identity insert On

Hi|||Hi Ajaxrand,

If you have access to enterprise manager, you can browse to the table, right click => design.

In the definition of the table, select the column, set identity on and seed value to the last existing value in the table. Do save.

Next record inserted into to the table will have the next identity column auto incremented.

Obviously take a copy of the table fist and test before changing in the production environment :)

Regards Purple|||

Quote:

Originally Posted by

If you have access to enterprise manager, you can browse to the table, right click => design.

In the definition of the table, select the column, set identity on and seed value to the last existing value in the table. Do save.


I could reach to this step Mytable >> Design Table
But There is nothing called set identity on or seed the value foo bar.|||Hi ajaxrand,

highlight the column (actually a row in this presentation) you are interested in by left clicking the grey square to the left of the column name. With the row highlighted the column detail will be shown in the bottom half of the window with all the things you need.

Regards Purple|||

Quote:

Originally Posted by Purple

Hi ajaxrand,

highlight the column (actually a row in this presentation) you are interested in by left clicking the grey square to the left of the column name. With the row highlighted the column detail will be shown in the bottom half of the window with all the things you need.

Regards Purple


Gotcha, Thanks purple.

I have another Quiz. the Original table is coming from MsAccess and i converted it to MSSQL using import export utilty.
There were nearly 2000 records in the table.with the changes i made to the Table structure (Identity Column) will it change those values in the table.|||Hi Ajaxrand,

as long as you don't change the datatype the identity changes we discussed will not change the data in that column on the table.

I would always suggest to run it in a development environment and check it out first anyway..

Good luck !

Purple|||Hi,

Thanks Purple.
Thanks gpinetto

Regards,
-Ajaxrand