爬行和抓取 iTunes App Store

发布于 2024-09-06 12:42:16 字数 480 浏览 2 评论 0原文

我注意到 iTunes 预览 允许您抓取并通过 http:// 协议抓取页面。但是,许多链接尝试在 iTunes 而不是浏览器中打开。例如,当您转到 iBooks 页面,它会立即尝试使用 itms:// 协议打开网址。

还有其他方法可以抓取 App Store 还是这是唯一的方法?

itms:// 协议链接本身可以以某种方式被抓取吗?

I noticed that iTunes preview allows you to crawl and scrape pages via the http:// protocol. However, many of the links are trying to be opened in iTunes rather than the browser. For example, when you go to the iBooks page, it immediately tries opening a url with an itms:// protocol.

Are there any other methods of crawling the App Store or is this the only way?

Can the itms:// protocol links themselves be crawled somehow?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

£烟消云散 2024-09-13 12:42:16

我会好好看看 iTunes Search API 和 iTunes Enterprise Partner API

您可能会以良好的 JSON 文件格式获得所需的大部分/全部信息。

如果您无法通过 API 获取所需的信息,我会感兴趣它是什么:)

I would have a decent look at the iTunes Search API and the iTunes Enterprise Partner API

You might get most/all of the information you need in a nice JSON file format.

If you can't get the information you need with the API, I would be interested what it is :)

月隐月明月朦胧 2024-09-13 12:42:16

正如 philipp 提到的,iTunes 搜索 API 是一种以 JSON 格式检索 App Store 列表数据的简单方法。

只需使用您的应用程序 ID 进行查询(您可以通过在 itunes.apple.com 上查看应用程序的 Web 列表来找到应用程序 ID),例如:

http://itunes.apple.com/lookup?id=INSERT_YOUR_APP_ID_HERE

然后将生成的 JSON 解析为您想要的内容。

As phillipp mentioned, the iTunes search API is an easy way to retrieve data about your App Store listings in JSON format.

Simply query for this with your app id (you can find the app id by viewing the web listing for your app at itunes.apple.com), ex:

http://itunes.apple.com/lookup?id=INSERT_YOUR_APP_ID_HERE

then, parse the resulting JSON to your heart's content.

如梦亦如幻 2024-09-13 12:42:16

http:// 链接和 itms:// 链接之间的唯一区别是,您需要将用户代理设置为 iTunes 用户代理,并且根据版本的不同,您可能还必须包含基于某些非验证码的验证码。 -如此秘密的算法。

例如,这是 iTunes 9 的代码:

# Some magic. Generates a seed we use for X-Apple-Validation. Adapted from LWP::UserAgent::iTMS_Client.
function comp_seed($url, $user_agent) {
    $random  = sprintf( "%04X%04X", rand(0,0x10000), rand(0,0x10000) );
    $static  = base64_decode("ROkjAaKid4EUF5kGtTNn3Q==");
    $url_end = ( preg_match("|.*/.*/.*(/.+)$|",$url,$matches)) ? $matches[1] : '?';
    $digest  = md5(join("",array($url_end, $user_agent, $static, $random)) );
    return $random . '-' . strtoupper($digest);
}

但是,如果您只是进行抓取,iTunes 预览应该可以满足您的目的,您向我们提供的 iBooks 页面链接有足够的信息可供抓取。

The only difference between http:// links and itms:// links is that you need to set your User-Agent to an iTunes user-agent, and depending on the version you may also have to include a verification code based on some not-so-secret algorithm.

For example this is the code for iTunes 9:

# Some magic. Generates a seed we use for X-Apple-Validation. Adapted from LWP::UserAgent::iTMS_Client.
function comp_seed($url, $user_agent) {
    $random  = sprintf( "%04X%04X", rand(0,0x10000), rand(0,0x10000) );
    $static  = base64_decode("ROkjAaKid4EUF5kGtTNn3Q==");
    $url_end = ( preg_match("|.*/.*/.*(/.+)$|",$url,$matches)) ? $matches[1] : '?';
    $digest  = md5(join("",array($url_end, $user_agent, $static, $random)) );
    return $random . '-' . strtoupper($digest);
}

However if you are only scraping, iTunes preview should work for your purposes, the link you gave us to the iBooks page had more than enough information to scrape.

谈下烟灰 2024-09-13 12:42:16

大约一年前,我们也尝试过给自己刮伤,但结果却变得非常令人头痛。菲利普的评论是一个很好的评论,因为苹果的企业提要(需要以合法用途申请它)确实有大量有用的信息,您可能会在抓取时找到这些信息。

有一些公司也提供数据即服务 - abto 和 AppMonsta 是我在寻找时听说的两家公司。我似乎找不到 abto 了,但 http://appmonsta.com 似乎是。搜索 API 看起来不错(从未尝试过)但有限制。

祝你好运!

We tried scraping ourselves too about a year ago and it just became too much of a headache. Philipp's comment is a good one as the enterprise feed from apple (need to apply for it with a legitimate use) does have a good amount of useful info that you might be after in scraping.

There are a few companies that offer data as a service too - abto and AppMonsta are two I heard of when I was looking. I can't seem to find abto anymore but http://appmonsta.com seems to be. The search API looks ok (never experimented) but limited.

Good luck!

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文