sqlite3.erationalerror:重复列名称:操作员
我一直在关注保罗·克里卡德(Paul Crickard)的书 - “与Python的数据工程”。我想安装和配置Apache气流。安装它后,我尝试用它来初始化数据库,
airflow db init
从而引发了以下错误:
[2022-06-28 10:40:08,549] {db.py:1448} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, Add ``operator`` and ``queued_dttm`` to ``task_instance`` table
Traceback (most recent call last):
File "/home/kien/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_context
self.dialect.do_execute(
File "/home/kien/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: duplicate column name: operator
我尚未尝试任何有意义的方法。我也不知道在哪里可以找到“ 13EB55F81627”文件。
我正在使用Ubuntu 20.04.04 LTS和Python 3.8.6。
同样,PIP冻结给了我:
alembic==1.8.0
amqp==5.1.1
anyio==3.6.1
apache-airflow==2.3.2
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-ftp==3.0.0
apache-airflow-providers-http==3.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-postgres==5.0.0
apache-airflow-providers-slack==5.0.0
apache-airflow-providers-sqlite==3.0.0
apispec==3.3.2
appdirs==1.4.4
apt-xapian-index==0.49
apturl==0.5.2
argcomplete==2.0.0
argon2-cffi==20.1.0
asn1crypto==0.24.0
async-generator==1.10
atomicwrites==1.1.5
attrs==20.3.0
Automat==0.8.0
Babel==2.10.3
backcall==0.2.0
beautifulsoup4==4.8.2
billiard==3.6.4.0
bleach==3.2.1
blinker==1.4
Brlapi==0.7.0
cachelib==0.9.0
cattrs==1.10.0
celery==5.2.7
certifi==2019.11.28
cffi==1.14.3
chardet==3.0.4
Click==7.0
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
clickclick==20.10.2
colorama==0.4.3
colorlog==4.8.0
command-not-found==0.3
commonmark==0.9.1
configobj==5.0.6
connexion==2.14.0
constantly==15.1.0
cron-descriptor==1.2.24
croniter==1.3.5
cryptography==2.8
cssselect==1.1.0
cupshelpers==1.0
cycler==0.10.0
Cython==0.29.21
dbus-python==1.2.16
decorator==4.4.2
defer==1.0.6
defusedxml==0.6.0
deluge==2.0.3
Deprecated==1.2.13
dill==0.3.3
distlib==0.3.1
distro==1.4.0
distro-info===0.23ubuntu1
dlx==1.0.4
dnspython==2.2.1
docplex==2.15.194
docutils==0.18.1
email-validator==1.2.1
entrypoints==0.3
et-xmlfile==1.0.1
fastdtw==0.3.4
fastjsonschema==2.15.0
filelock==3.6.0
Flask==1.1.4
Flask-AppBuilder==3.4.5
Flask-Babel==2.0.0
Flask-Caching==1.11.1
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.3.0
Flask-Session==0.4.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
flower==1.0.0
gpg===1.13.1-unknown
graphviz==0.20
greenlet==1.1.2
gunicorn==20.1.0
h11==0.12.0
h5py==3.1.0
html5lib==1.0.1
httpcore==0.15.0
httplib2==0.14.0
httpx==0.23.0
humanize==4.2.2
hyperlink==19.0.0
idna==2.8
importlib-metadata==4.12.0
importlib-resources==5.8.0
incremental==16.10.1
inflection==0.5.1
intel-openmp==2022.0.2
ipykernel==5.3.4
ipython==7.19.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
itsdangerous==1.1.0
jdcal==1.0
jedi==0.17.2
Jinja2==2.11.2
joblib==0.17.0
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.7
jupyter-console==6.2.0
jupyter-core==4.7.0
jupyterlab-pygments==0.1.2
keyring==18.0.1
kiwisolver==1.0.1
kombu==5.2.4
language-selector==0.1
launchpadlib==1.10.13
lazr.restfulclient==0.14.2
lazr.uri==1.0.3
lazy-object-proxy==1.7.1
lockfile==0.12.2
louis==3.12.0
lxml==4.6.2
macaroonbakery==1.3.1
Mako==1.1.0
Markdown==3.3.7
MarkupSafe==2.0.1
marshmallow==3.17.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
matplotlib==3.1.2
mistune==0.8.4
mkl==2022.0.2
more-itertools==4.2.0
mpmath==1.2.1
multitasking==0.0.9
nbclient==0.5.1
nbconvert==6.0.7
nbformat==5.0.8
nest-asyncio==1.4.3
netifaces==0.10.4
networkx==2.5
notebook==6.1.5
ntlm-auth==1.5.0
numexpr==2.7.1
numpy==1.17.4
oauthlib==3.1.0
olefile==0.46
openpyxl==3.0.3
packaging==20.3
pandas==1.3.3
pandocfilters==1.4.3
parso==0.7.1
pathspec==0.9.0
pendulum==2.1.2
pexpect==4.6.0
pickleshare==0.7.5
Pillow==7.0.0
platformdirs==2.5.1
pluggy==1.0.0
ply==3.11
prison==0.2.1
prometheus-client==0.9.0
prompt-toolkit==3.0.8
protobuf==3.6.1
psutil==5.5.1
psycopg2-binary==2.9.3
ptyprocess==0.6.0
py==1.8.1
pyasn1==0.4.2
pyasn1-modules==0.2.1
pybind11==2.6.2
pycairo==1.16.2
pycparser==2.20
pycrypto==2.6.1
pycups==1.9.73
pydot==1.4.2
Pygments==2.7.2
PyGObject==3.36.0
PyHamcrest==1.9.0
PyJWT==1.7.1
pylatexenc==2.8
pymacaroons==0.13.0
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==2.4.6
PyQt5==5.14.1
pyRFC3339==1.1
pyrsistent==0.17.3
pytest==4.6.9
python-apt==2.0.0+ubuntu0.20.4.7
python-constraint==1.4.0
python-daemon==2.3.0
python-dateutil==2.8.1
python-debian===0.1.36ubuntu1
python-libtorrent==1.1.13
python-nvd3==0.15.0
python-slugify==6.1.2
python3-openid==3.2.0
pytz==2022.1
pytzdata==2020.1
pyxdg==0.26
PyYAML==5.3.1
pyzmq==20.0.0
qiskit==0.23.6
qiskit-aer==0.7.5
qiskit-aqua==0.8.2
qiskit-ibmq-provider==0.11.1
qiskit-ignis==0.5.2
qiskit-terra==0.16.4
qiskit-textbook==0.1.0
qtconsole==4.7.7
QtPy==1.9.0
Quandl==3.6.0
ranger-fm==1.9.3
rencode==1.0.6
reportlab==3.5.34
requests==2.22.0
requests-ntlm==1.1.0
requests-unixsocket==0.2.0
retworkx==0.7.2
rfc3986==1.5.0
rich==12.4.4
scikit-learn==1.0.2
scipy==1.3.3
scour==0.37
seaborn==0.11.1
SecretStorage==2.3.1
Send2Trash==1.5.0
service-identity==18.1.0
setproctitle==1.1.10
simplejson==3.16.0
sip==4.19.21
six==1.14.0
slack-sdk==3.17.2
sniffio==1.2.0
soupsieve==1.9.5
SQLAlchemy==1.4.9
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.38.2
ssh-import-id==5.10
swagger-ui-bundle==0.0.9
sympy==1.7.1
system-service==0.3
systemd-python==234
tables==3.6.1
tabulate==0.8.10
tbb==2021.5.1
tenacity==8.0.1
termcolor==1.1.0
terminado==0.9.1
testpath==0.4.4
text-unidecode==1.3
threadpoolctl==2.1.0
tornado==6.1
traitlets==5.0.5
Twisted==18.9.0
typing-extensions==4.2.0
ubuntu-advantage-tools==27.8
ubuntu-drivers-common==0.0.0
ufw==0.36
unattended-upgrades==0.1
unicodecsv==0.14.1
urllib3==1.25.8
usb-creator==0.3.7
vine==5.0.0
virtualenv==20.14.0
wadllib==1.3.3
wcwidth==0.1.8
webencodings==0.5.1
websockets==8.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wrapt==1.14.1
WTForms==2.3.3
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yfinance==0.1.55
zipp==3.8.0
zope.interface==4.7.1
我试图通过执行以下操作来摆脱重复的列:
sqlite> .output text.txt
sqlite> .open airflow.db
sqlite> .dump
清空了airflow.db,将其转动到气流时
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
COMMIT;
。
airflow db init
它 )。初始化完成。所以我的问题是有人知道我可能做了什么吗?
I have been following the book of Paul Crickard - "Data Engineering with python". I wanted to install and configure Apache Airflow. After installing it, I tried to initialize the database with
airflow db init
It raised the following error:
[2022-06-28 10:40:08,549] {db.py:1448} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, Add ``operator`` and ``queued_dttm`` to ``task_instance`` table
Traceback (most recent call last):
File "/home/kien/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_context
self.dialect.do_execute(
File "/home/kien/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: duplicate column name: operator
I have not yet tried anything meaningful. I also do not know where I can find the file "13eb55f81627".
I am using Ubuntu 20.04.04 LTS and python 3.8.6.
Also pip freeze gave me:
alembic==1.8.0
amqp==5.1.1
anyio==3.6.1
apache-airflow==2.3.2
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-ftp==3.0.0
apache-airflow-providers-http==3.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-postgres==5.0.0
apache-airflow-providers-slack==5.0.0
apache-airflow-providers-sqlite==3.0.0
apispec==3.3.2
appdirs==1.4.4
apt-xapian-index==0.49
apturl==0.5.2
argcomplete==2.0.0
argon2-cffi==20.1.0
asn1crypto==0.24.0
async-generator==1.10
atomicwrites==1.1.5
attrs==20.3.0
Automat==0.8.0
Babel==2.10.3
backcall==0.2.0
beautifulsoup4==4.8.2
billiard==3.6.4.0
bleach==3.2.1
blinker==1.4
Brlapi==0.7.0
cachelib==0.9.0
cattrs==1.10.0
celery==5.2.7
certifi==2019.11.28
cffi==1.14.3
chardet==3.0.4
Click==7.0
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
clickclick==20.10.2
colorama==0.4.3
colorlog==4.8.0
command-not-found==0.3
commonmark==0.9.1
configobj==5.0.6
connexion==2.14.0
constantly==15.1.0
cron-descriptor==1.2.24
croniter==1.3.5
cryptography==2.8
cssselect==1.1.0
cupshelpers==1.0
cycler==0.10.0
Cython==0.29.21
dbus-python==1.2.16
decorator==4.4.2
defer==1.0.6
defusedxml==0.6.0
deluge==2.0.3
Deprecated==1.2.13
dill==0.3.3
distlib==0.3.1
distro==1.4.0
distro-info===0.23ubuntu1
dlx==1.0.4
dnspython==2.2.1
docplex==2.15.194
docutils==0.18.1
email-validator==1.2.1
entrypoints==0.3
et-xmlfile==1.0.1
fastdtw==0.3.4
fastjsonschema==2.15.0
filelock==3.6.0
Flask==1.1.4
Flask-AppBuilder==3.4.5
Flask-Babel==2.0.0
Flask-Caching==1.11.1
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.3.0
Flask-Session==0.4.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
flower==1.0.0
gpg===1.13.1-unknown
graphviz==0.20
greenlet==1.1.2
gunicorn==20.1.0
h11==0.12.0
h5py==3.1.0
html5lib==1.0.1
httpcore==0.15.0
httplib2==0.14.0
httpx==0.23.0
humanize==4.2.2
hyperlink==19.0.0
idna==2.8
importlib-metadata==4.12.0
importlib-resources==5.8.0
incremental==16.10.1
inflection==0.5.1
intel-openmp==2022.0.2
ipykernel==5.3.4
ipython==7.19.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
itsdangerous==1.1.0
jdcal==1.0
jedi==0.17.2
Jinja2==2.11.2
joblib==0.17.0
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.7
jupyter-console==6.2.0
jupyter-core==4.7.0
jupyterlab-pygments==0.1.2
keyring==18.0.1
kiwisolver==1.0.1
kombu==5.2.4
language-selector==0.1
launchpadlib==1.10.13
lazr.restfulclient==0.14.2
lazr.uri==1.0.3
lazy-object-proxy==1.7.1
lockfile==0.12.2
louis==3.12.0
lxml==4.6.2
macaroonbakery==1.3.1
Mako==1.1.0
Markdown==3.3.7
MarkupSafe==2.0.1
marshmallow==3.17.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
matplotlib==3.1.2
mistune==0.8.4
mkl==2022.0.2
more-itertools==4.2.0
mpmath==1.2.1
multitasking==0.0.9
nbclient==0.5.1
nbconvert==6.0.7
nbformat==5.0.8
nest-asyncio==1.4.3
netifaces==0.10.4
networkx==2.5
notebook==6.1.5
ntlm-auth==1.5.0
numexpr==2.7.1
numpy==1.17.4
oauthlib==3.1.0
olefile==0.46
openpyxl==3.0.3
packaging==20.3
pandas==1.3.3
pandocfilters==1.4.3
parso==0.7.1
pathspec==0.9.0
pendulum==2.1.2
pexpect==4.6.0
pickleshare==0.7.5
Pillow==7.0.0
platformdirs==2.5.1
pluggy==1.0.0
ply==3.11
prison==0.2.1
prometheus-client==0.9.0
prompt-toolkit==3.0.8
protobuf==3.6.1
psutil==5.5.1
psycopg2-binary==2.9.3
ptyprocess==0.6.0
py==1.8.1
pyasn1==0.4.2
pyasn1-modules==0.2.1
pybind11==2.6.2
pycairo==1.16.2
pycparser==2.20
pycrypto==2.6.1
pycups==1.9.73
pydot==1.4.2
Pygments==2.7.2
PyGObject==3.36.0
PyHamcrest==1.9.0
PyJWT==1.7.1
pylatexenc==2.8
pymacaroons==0.13.0
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==2.4.6
PyQt5==5.14.1
pyRFC3339==1.1
pyrsistent==0.17.3
pytest==4.6.9
python-apt==2.0.0+ubuntu0.20.4.7
python-constraint==1.4.0
python-daemon==2.3.0
python-dateutil==2.8.1
python-debian===0.1.36ubuntu1
python-libtorrent==1.1.13
python-nvd3==0.15.0
python-slugify==6.1.2
python3-openid==3.2.0
pytz==2022.1
pytzdata==2020.1
pyxdg==0.26
PyYAML==5.3.1
pyzmq==20.0.0
qiskit==0.23.6
qiskit-aer==0.7.5
qiskit-aqua==0.8.2
qiskit-ibmq-provider==0.11.1
qiskit-ignis==0.5.2
qiskit-terra==0.16.4
qiskit-textbook==0.1.0
qtconsole==4.7.7
QtPy==1.9.0
Quandl==3.6.0
ranger-fm==1.9.3
rencode==1.0.6
reportlab==3.5.34
requests==2.22.0
requests-ntlm==1.1.0
requests-unixsocket==0.2.0
retworkx==0.7.2
rfc3986==1.5.0
rich==12.4.4
scikit-learn==1.0.2
scipy==1.3.3
scour==0.37
seaborn==0.11.1
SecretStorage==2.3.1
Send2Trash==1.5.0
service-identity==18.1.0
setproctitle==1.1.10
simplejson==3.16.0
sip==4.19.21
six==1.14.0
slack-sdk==3.17.2
sniffio==1.2.0
soupsieve==1.9.5
SQLAlchemy==1.4.9
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.38.2
ssh-import-id==5.10
swagger-ui-bundle==0.0.9
sympy==1.7.1
system-service==0.3
systemd-python==234
tables==3.6.1
tabulate==0.8.10
tbb==2021.5.1
tenacity==8.0.1
termcolor==1.1.0
terminado==0.9.1
testpath==0.4.4
text-unidecode==1.3
threadpoolctl==2.1.0
tornado==6.1
traitlets==5.0.5
Twisted==18.9.0
typing-extensions==4.2.0
ubuntu-advantage-tools==27.8
ubuntu-drivers-common==0.0.0
ufw==0.36
unattended-upgrades==0.1
unicodecsv==0.14.1
urllib3==1.25.8
usb-creator==0.3.7
vine==5.0.0
virtualenv==20.14.0
wadllib==1.3.3
wcwidth==0.1.8
webencodings==0.5.1
websockets==8.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wrapt==1.14.1
WTForms==2.3.3
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yfinance==0.1.55
zipp==3.8.0
zope.interface==4.7.1
I have tried to get rid of the duplicate columns by doing the following:
sqlite> .output text.txt
sqlite> .open airflow.db
sqlite> .dump
it emptied the airflow.db and when turning it to airflow.sql it gave:
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
COMMIT;
Re-running
airflow db init
finally did something (no additional errors were raised). Initialization done. So my question is does anyone know what I have possibly done?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
对我来说,该解决方案是从气流文件夹中删除airflow.db,并再次运行
airflow startalone
命令。For me, the solution was deleting the airflow.db from the airflow folder and running the
airflow standalone
command again.