从 URL 导入数据

发布于 2024-12-23 18:49:31 字数 917 浏览 1 评论 0原文

圣路易斯联邦储备银行的各种网页上提供了大量数据,例如:

http://research.stlouisfed.org/fred2/series/OILPRICE/downloaddata?cid=32217 http://www.federalreserve.gov/releases/h10/summary/default.htm http://research.stlouisfed.org/fred2/series/DGS20

数据集会更新,有些会经常更新和日常一样。我倾向于对每日数据感兴趣(请参阅 URL 上的上述设置)

我想将这些类型的价格或费率数据流(可通过上述 URL 以 CSV 或 Excel 文件形式访问)直接导入到 Mathematica 中。

我查看了有关 Importing[] 的文档,但发现关于如何进行此类操作的文档很少(实际上没有)。

看起来我需要导航到页面,发送一些数据以选择特定文件和格式,触发下载,然后从我自己的计算机访问下载的数据。如果我可以直接从网站访问数据就更好了。

我曾希望 Wolfram Alpha 能让这类事情变得简单,但我还没有取得任何成功。

FinancialData[] 对于这类事情来说似乎很自然,但我无论如何也不认为这样做。财务数据有很多功能,但我看不出有什么方法可以得到这种东西。

有谁有这方面的经验或者有人可以指出我正确的方向吗?

The St. Louis Federal Reserve Bank has a great set of data available on a variety of their web pages, such as:

http://research.stlouisfed.org/fred2/series/OILPRICE/downloaddata?cid=32217
http://www.federalreserve.gov/releases/h10/summary/default.htm
http://research.stlouisfed.org/fred2/series/DGS20

The data sets get updated, some as often as daily. I tend to have an interest in the daily data (see the above settings on the URLS)

I'd like to import these kinds of price or rate data streams (accessible as CSV or Excel files at the above URLs) directly into Mathematica.

I've looked at the documentation on Importing[] but I find scant documentation (actually none) on how to go about something like this.

It looks like I need to navigate to the pages, send some data to select specific files and formats, trigger the download, then access the downloaded data from my own machine. Even better if I could access the data directly from the sites.

I had hoped Wolfram Alpha might make this sort thing easy, but I haven't had any success.

FinancialData[] would seem natural for this sort of thing, but I don't see anyway to do it. Financial data has lots of features, but I don't see a way yo get this sort of thing.

Does anyone have any experience with this or can someone point me in the right direction?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

征棹 2024-12-30 18:49:31

您可以直接从 URL 导入。例如,可以按如下方式获取和可视化来自 federation.gov 的数据。

url = "http://www.federalreserve.gov/datadownload/Output.aspx?";
url = url<>"rel=H10&series=a660e724c705cea4b7bd1d1b85789862&lastObs=&";
url = url<>"from=&to=&filetype=csv&label=include&layout=seriescolumn";
data = Import[url, "CSV"];
DateListPlot[data[[7 ;;]], Joined -> True]

为了方便起见,我将 url 分解了,因为它太长了。在我确切地知道如何绘制数据之前,我必须检查数据的内容 - 这通常是必要的步骤。我确信可以通过类似的方式获取来自 stlouisfed.org 的数据,但它需要使用带有密钥的 API 来访问它。

You can Import directly from a URL. For example, the data from federalreserve.gov can be obtained and visualized as follows.

url = "http://www.federalreserve.gov/datadownload/Output.aspx?";
url = url<>"rel=H10&series=a660e724c705cea4b7bd1d1b85789862&lastObs=&";
url = url<>"from=&to=&filetype=csv&label=include&layout=seriescolumn";
data = Import[url, "CSV"];
DateListPlot[data[[7 ;;]], Joined -> True]

I broke up url for convenience, since it's so long. I had to examine the contents of data before I knew exactly how to plot it - a step that is typically necessary. I'm sure that the data from stlouisfed.org can be obtained in a similar way, but it requires the use of an API with key to access it.

金兰素衣 2024-12-30 18:49:31

正如 Mark 所说,您可以直接从 URL 获取数据。您的石油数据可以从与您之前不同的 URL 导入:

http://research.stlouisfed .org/fred2/data/OILPRICE.txt

使用该 URL,您可以执行以下操作:

oil = Import["http://research.stlouisfed.org/fred2/data/OILPRICE.txt",
"Table", "HeaderLines" -> 12, "DateStringFormat" -> {"Year", "Month", "Day"}];
DateListPlot[oil, Joined -> True, PlotRange -> All]

请注意,“HeaderLines”->12 选项会去除前 12 行中的标题文本(您必须计算标题行才能知道要删除多少行)。我还指定了日期格式。

要查找该 URL,请像之前一样执行操作,但单击数据系列,然后在看到图表时从左侧菜单中选择“查看数据”。

As Mark said, you can get the data directly from a URL. Your oil data can be imported from a different URL than you had:

http://research.stlouisfed.org/fred2/data/OILPRICE.txt

With that URL, you can do this:

oil = Import["http://research.stlouisfed.org/fred2/data/OILPRICE.txt",
"Table", "HeaderLines" -> 12, "DateStringFormat" -> {"Year", "Month", "Day"}];
DateListPlot[oil, Joined -> True, PlotRange -> All]

Note that "HeaderLines"->12 option strips off the header text in the first 12 lines (you have to count the header lines to know how many to remove). I've also specified the date format.

To find that URL, do as you did before, but click on a data series and then choose View Data from the menu on the left when you see the chart.

眸中客 2024-12-30 18:49:31

该文档有一个关于从网页中提取数据的简短示例:

http://reference.wolfram.com/mathematica/ howto/CleanUpDataImportedFromAWebsite.html

当然,实际需要完成的操作因页面而异。

The documentation has a short example on extracting data out of a webpage:

http://reference.wolfram.com/mathematica/howto/CleanUpDataImportedFromAWebsite.html

Of course, what actually needs to be done will vary significantly from page to page.

自我难过 2024-12-30 18:49:31

关于如何使用 API 密钥执行此操作的讨论:

http://library.wolfram.com /infocenter/MathSource/7583/

该函数基于API文档。我已经有几年没有看过代码了,根据记忆,我很快就把它组合在一起,但我已经定期使用它超过 2 年了,没有出现任何问题。以下是从 1992 年初至今每月非季节性调整零售额的示例:

在此处输入图像描述

Wolfram alpha 也使用 FRED数据,因此您可以使用它作为直接导入的替代方案,但正确获取查询更加棘手。我更喜欢直接使用FRED。另外,根据内存,数据仅在发布后的第二天在 alpha 版本中可用,这不是您通常想要的。

在此处输入图像描述

discussion on how to do this with your API key here:

http://library.wolfram.com/infocenter/MathSource/7583/

the function is based on the API documentation. I haven't looked at the code for a couple of years and from memory I put it together rather quickly but I have used it regularly for over 2 years without problems. Here is an example for monthly non seasonally adjusted retail sales from early 1992 to now:

enter image description here

wolfram alpha also uses FRED data so you could use that as an alternative to direct import but it is more tricky to get the query right. I prefer to use FRED directly. Also from memory the data is only available on alpha the day after the release, which is not what you would typically want.

enter image description here

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文