如何将 PostgreSQL 数据库迁移到 SQLServer 数据库?

发布于 2024-11-18 09:59:57 字数 140 浏览 1 评论 0原文

我有一个 PostgreSQL 数据库,我想将其迁移到 SQL Server - 架构和数据。我很穷,所以我不想付钱。我也很懒,所以不想做太多工作。目前我正在逐个桌子做这个,大约有100个桌子要做。这是极其乏味的。

有什么技巧可以达到我想要的效果吗?

I have a PostgreSQL database that I want to move to SQL Server -- both schema and data. I am poor so I don't want to pay any money. I am also lazy, so I don't want to do very much work. Currently I'm doing this table by table, and there are about 100 tables to do. This is extremely tedious.

Is there some sort of trick that does what I want?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

佞臣 2024-11-25 09:59:58

我找到了一种更快、更简单的方法来实现这一目标。

首先将表(或查询)复制到制表符分隔文件,如下所示:

COPY (SELECT siteid, searchdist, listtype, list, sitename, county, street, 
   city, state, zip, georesult, elevation, lat, lng, wkt, unlocated_bool, 
   id, status, standard_status, date_opened_or_reported, date_closed, 
   notes, list_type_description FROM mlocal) TO 'c:\SQLAzureImportFiles\data_script_mlocal.tsv' NULL E''

接下来,您需要在 SQL 中创建表,这不会为您处理任何架构。该架构必须在字段顺序和数据类型上与导出的 tsv 文件相匹配。

最后,您运行 SQL 的 bcp 实用程序来引入 tsv 文件,如下所示:

bcp MyDb.dbo.mlocal in "\\NEWDBSERVER\SQLAzureImportFiles\data_script_mlocal.tsv" -S tcp:YourDBServer.database.windows.net -U YourUserName -P YourPassword -c

我遇到的一些值得注意的事情。 Postgres 和 SQL Server 处理布尔字段的方式不同。您的 SQL Server 架构需要将布尔字段设置为 varchar(1),结果数据将为“f”、“t”或 null。然后您必须将该字段转换为位。做类似的事情:

ALTER TABLE mlocal ADD unlocated bit;
UPDATE mlocal SET unlocated=1 WHERE unlocated_bool='t';
UPDATE mlocal SET unlocated=0 WHERE unlocated_bool='f';
ALTER TABLE mlocal DROP COLUMN unlocated_bool;

另一件事是两个平台之间的地理/几何字段非常不同。使用 ST_AsText(geo) 将几何字段导出为 WKT,并在 SQL Server 端进行适当转换。

可能还有更多的不兼容性需要像这样的调整。

编辑。因此,虽然这种技术在技术上确实可行,但我试图将数百万条记录从 100 多个表传输到 SQL Azure,但事实证明,bcp 到 SQL Azure 非常不稳定。我不断收到间歇性无法打开 BCP 主机数据文件错误,服务器间歇性超时,并且由于某种原因,某些记录未传输,没有任何错误或问题的迹象。因此,这种技术对于将大量数据传输到 Azure SQL 来说并不稳定。

I have found a faster and easier way to accomplish this.

First copy your table (or query) to a tab delimited file like so:

COPY (SELECT siteid, searchdist, listtype, list, sitename, county, street, 
   city, state, zip, georesult, elevation, lat, lng, wkt, unlocated_bool, 
   id, status, standard_status, date_opened_or_reported, date_closed, 
   notes, list_type_description FROM mlocal) TO 'c:\SQLAzureImportFiles\data_script_mlocal.tsv' NULL E''

Next you need to create your table in SQL, this will not handle any schema for you. The schema must match your exported tsv file in field order and data types.

Finally you run SQL's bcp utility to bring in the tsv file like so:

bcp MyDb.dbo.mlocal in "\\NEWDBSERVER\SQLAzureImportFiles\data_script_mlocal.tsv" -S tcp:YourDBServer.database.windows.net -U YourUserName -P YourPassword -c

A couple of things of note that I encountered. Postgres and SQL Server handle boolean fields differently. Your SQL Server schema need to have your boolean fields set to varchar(1) and the resulting data will be 'f', 't' or null. You will then have to convert this field to a bit. doing something like:

ALTER TABLE mlocal ADD unlocated bit;
UPDATE mlocal SET unlocated=1 WHERE unlocated_bool='t';
UPDATE mlocal SET unlocated=0 WHERE unlocated_bool='f';
ALTER TABLE mlocal DROP COLUMN unlocated_bool;

Another thing is the geography/geometry fields are very different between the two platforms. Export the geometry fields as WKT using ST_AsText(geo) and convert appropriately on the SQL Server end.

There may be more incompatibilities needing tweaks like this.

EDIT. So whereas this technique does technically work, I am trying to transfer several million records from 100+ tables to SQL Azure and bcp to SQL Azure is pretty flaky it turns out. I keep getting intermittent Unable to open BCP host data-file errors, the server is intermittently timing out and for some reason some records are not getting transferred with no indications of errors or problems. So this technique is not stable for transferring large amounts of data to Azure SQL.

神魇的王 2024-11-25 09:59:58
If you want to migrate only data and table schema from PostgreSQL to MS SQL

Step 1: Download Dbeavre. https://dbeaver.io/download/

Step 2: Connect the SQL server and PostgreSQL server.

Step 3: Create the Database in the SQL server.

Step 4: After that select the table which you want to migrate select all tables from PostgreSQL and then Right Click-->Export Data-->Database(Click Next)-->Select Target Database using Configure Next and finally you will proceed.
https://i.sstatic.net/v8eTqNqo.png
https://i.sstatic.net/oMJxNgA4.png
https://i.sstatic.net/zOx0L725.png
https://i.sstatic.net/CUAFIFfr.png
https://i.sstatic.net/KxDfdSGy.png
If you want to migrate only data and table schema from PostgreSQL to MS SQL

Step 1: Download Dbeavre. https://dbeaver.io/download/

Step 2: Connect the SQL server and PostgreSQL server.

Step 3: Create the Database in the SQL server.

Step 4: After that select the table which you want to migrate select all tables from PostgreSQL and then Right Click-->Export Data-->Database(Click Next)-->Select Target Database using Configure Next and finally you will proceed.
https://i.sstatic.net/v8eTqNqo.png
https://i.sstatic.net/oMJxNgA4.png
https://i.sstatic.net/zOx0L725.png
https://i.sstatic.net/CUAFIFfr.png
https://i.sstatic.net/KxDfdSGy.png
も让我眼熟你 2024-11-25 09:59:57

您应该能够在此 Serverfault 页面中接受的答案中找到一些有用的信息:https://serverfault.com/questions/65407/将 postgresql 数据库迁移到 ms-sql-2005 的最佳工具

如果您可以在没有数据的情况下转换架构,则可以使用以下命令缩短数据的步骤:

pg_dump --data-only --column-inserts your_db_name > data_load_script.sql

此加载将非常慢,但 --column-inserts 选项会生成每行数据可能使用最通用的 INSERT 语句,并且应该兼容。

编辑:关于转换模式的建议如下:

我将首先转储模式,但删除与所有权或权限有关的任何内容。这应该足够了:

pg_dump --schema-only --no-owner --no-privileges your_db_name > schema_create_script.sql

编辑此文件以将 BEGIN TRANSACTION; 行添加到开头,将 ROLLBACK TRANSACTION; 添加到末尾。现在您可以加载它并在 SQL Server 的查询窗口中运行它。如果出现任何错误,请确保转到文件底部,突出显示 ROLLBACK 语句并运行它(在突出显示语句时按 F5)。

基本上,您必须解决每个错误,直到脚本干净地运行为止。然后,您可以将 ROLLBACK TRANSACTION 更改为 COMMIT TRANSACTION 并最后运行一次。

不幸的是,我无法帮助您解决您可能会看到的错误,因为我从未从 PostgreSQL 转到 SQL Server,只有反过来。然而,我认为有些事情会成为一个问题(显然,这不是一个详尽的列表):

  • PostgreSQL 通过将 NOT NULL INTEGER 字段链接到 SEQUENCE 来自动递增字段> 使用DEFAULT。在 SQL Server 中,这是一个 IDENTITY 列,但它们并不完全相同。我不确定它们是否等效,但是如果您的原始架构充满了“id”字段,您可能会遇到一些麻烦。我不知道 SQL Server 是否有 CREATE SEQUENCE,因此您可能必须删除它们。
  • 数据库函数/存储过程不在 RDBMS 平台之间进行转换。您需要删除所有 CREATE FUNCTION 语句并手动转换算法。
  • 请注意数据文件的编码。我是 Linux 人员,所以我不知道如何在 Windows 中验证编码,但是您需要确保 SQL Server 期望的内容与您从 PostgreSQL 导入的文件相同。 pg_dump 有一个选项 --encoding=,可让您设置特定的编码。我似乎记得 Windows 倾向于使用两字节的 UTF-16 编码来表示 Unicode,而 PostgreSQL 使用 UTF-8。由于 UTF-16 输出,我在从 SQL Server 到 PostgreSQL 时遇到了一些问题,因此值得研究。
  • PostgreSQL 数据类型 TEXT 只是一个没有最大长度的 VARCHAR。在 SQL Server 中,TEXT 很复杂(并且已弃用)。原始架构中声明为 TEXT 的每个字段都需要检查是否有适当的 SQL Server 数据类型。
  • SQL Server 对于 UNICODE 数据有额外的数据类型。我对它不够熟悉,无法提出建议。我只是指出这可能是一个问题。

You should be able to find some useful information in the accepted answer in this Serverfault page: https://serverfault.com/questions/65407/best-tool-to-migrate-a-postgresql-database-to-ms-sql-2005.

If you can get the schema converted without the data, you may be able to shorten the steps for the data by using this command:

pg_dump --data-only --column-inserts your_db_name > data_load_script.sql

This load will be quite slow, but the --column-inserts option generates the most generic INSERT statements possible for each row of data and should be compatible.

EDIT: Suggestions on converting the schema follows:

I would start by dumping the schema, but removing anything that has to do with ownership or permissions. This should be enough:

pg_dump --schema-only --no-owner --no-privileges your_db_name > schema_create_script.sql

Edit this file to add the line BEGIN TRANSACTION; to the beginning and ROLLBACK TRANSACTION; to the end. Now you can load it and run it in a query window in SQL Server. If you get any errors, make sure you go to the bottom of the file, highlight the ROLLBACK statement and run it (by hitting F5 while the statement is highlighted).

Basically, you have to resolve each error until the script runs through cleanly. Then you can change the ROLLBACK TRANSACTION to COMMIT TRANSACTION and run one final time.

Unfortunately, I cannot help with which errors you may see as I have never gone from PostgreSQL to SQL Server, only the other way around. Some things that I would expect to be an issue, however (obviously, NOT an exhaustive list):

  • PostgreSQL does auto-increment fields by linking a NOT NULL INTEGER field to a SEQUENCE using a DEFAULT. In SQL Server, this is an IDENTITY column, but they're not exactly the same thing. I'm not sure if they are equivalent, but if your original schema is full of "id" fields, you may be in for some trouble. I don't know if SQL Server has CREATE SEQUENCE, so you may have to remove those.
  • Database functions / Stored Procedures do not translate between RDBMS platforms. You'll need to remove any CREATE FUNCTION statements and translate the algorithms manually.
  • Be careful about encoding of the data file. I'm a Linux person, so I have no idea how to verify encoding in Windows, but you need to make sure that what SQL Server expects is the same as the file you are importing from PostgreSQL. pg_dump has an option --encoding= that will let you set a specific encoding. I seem to recall that Windows tends to use two-byte, UTF-16 encoding for Unicode where PostgreSQL uses UTF-8. I had some issue going from SQL Server to PostgreSQL due to UTF-16 output so it would be worth researching.
  • The PostgreSQL datatype TEXT is simply a VARCHAR without a max length. In SQL Server, TEXT is... complicated (and deprecated). Each field in your original schema that are declared as TEXT will need to be reviewed for an appropriate SQL Server data type.
  • SQL Server has extra data types for UNICODE data. I'm not familiar enough with it to make suggestions. I'm just pointing out that it may be an issue.
风筝有风,海豚有海 2024-11-25 09:59:57

您可以使用 Navicate 这个功能强大的 GUI 工具来处理各种数据库,包括 Postgres 和 SQL Server。
您可以轻松传输架构和数据,如下所示:

  1. 为源数据库和目标数据库创建两个连接

在此处输入图像描述

  1. 转到工具 ->数据传输

选择源数据库和目标数据库及其 IP、数据库名称和架构

,如果目标表不存在,它将创建

Tada,需要 10 分钟才能将我的 63 个表及其数据从 Postgres 传输到 SQL Server。

好好享受!

You can use Navicate a powerful GUI tool for working with various databases including Postgres and SQL Server.
You can transfer both schema and data easily as follows:

  1. Create two connection for source and target database

enter image description here

  1. Go to Tools -> Data Transfer

Select source database and target database with its IP, database name and schema
enter image description here

as you can see in the option, if target table is not exist, it would create

Tada, it takes 10 mins to transfer whole my 63 tables and its data from Postgres to SQL Server.

Enjoy it!

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文