带有 Foreach 文件枚举器选项的 Foreach 循环容器将所有文件迭代两次

发布于 2024-11-25 17:59:51 字数 205 浏览 1 评论 0原文

我正在使用 SSIS Foreach 循环容器来迭代网络共享上具有特定模式的文件。

我遇到了循环容器的一种不可重现的故障:

有时循环会执行两次。处理完所有文件后,将从第一个文件开始。

有人遇到过类似的错误吗? 也许不是直接使用 SSIS 而是使用某种技术访问 Windows 共享上的文件? 此错误是否与某些网络问题有关?

谢谢。

I am using the SSIS Foreach Loop Container to iterate through files with a certain pattern on a network share.

I am encountering an kind of unreproducible malfunction of the Loop Container:

Sometimes the loop is executed twice. After all files were processed it starts over with the first file.

Have anyone encountered a similar bug?
Maybe not directly using SSIS but accessing files on a Windows share with some kind of technology?
Could this error relate to some network issues?

Thanks.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

凉风有信 2024-12-02 17:59:51

我发现在处理 Excel 文件并使用 *.xlsx 通配符驱动 ​​foreach 时就是这种情况。

一旦我将日志记录到位,我注意到当打开 Excel 时,它会生成一个前缀为 ~$ 的 Excel 文件。这是由 foreach 循环获取的。

所以我使用了类似于 http://geekswithblogs.net/Compudicted/archive/2012/01/11/the-ssis-expression-wayndashskipping-an-unwanted-file.aspx 排除带有 ~$ 的文件在文件名中。

I found this to be the case whilst working with Excel files and using the *.xlsx wildcard to drive the foreach.

Once I put logging in place I noticed that when the Excel was opened it produced an excel file prefixed with ~$. This was picked up by the foreach loop.

So I used a trick similar to http://geekswithblogs.net/Compudicted/archive/2012/01/11/the-ssis-expression-wayndashskipping-an-unwanted-file.aspx to exclude files with a ~$ in the filename.

笑叹一世浮沉 2024-12-02 17:59:51

您收到什么错误消息(SSIS 日志/Eventvwr 消息)?

与@Siva类似,我没有遇到过这个,但你可以使用一些想法来尝试和诊断。您可能已经在做其中的一些事情,为了完整起见,我刚刚将它们写下来......

  • 记录所有已处理的文件。将一行写入日志文件/表预处理(每个文件),然后进行后处理(每个文件)。保留每个文件的完整路径。这实际上是我们在 ETL 实现中作为标准所做的事情,因为用户经常会向我们询问有关何时加载/加载什么内容的问题。这将允许您查看文件是否实际上被处理了两次。
  • 也许尝试在处理每个文件后将其移动到不同的目录。这将使再次处理文件变得更加困难,并且问题可能会消失。 (如果您正在从“主”区域处理它们(因此无法移动它们),请考虑将文件复制到“等待”文件夹,然后处理它们并将它们移动到“已处理”文件夹)
  • @Siva's评论很有​​趣 - 查看“遍历子文件夹”复选框。
  • 检查您的 eventvwr 是否有奇怪的网络事件或应用程序事件(SQL Server 重新启动?)
  • 使用 perf mon 来查看服务器上的网络负载是否有任何奇怪的情况发生(有点随机的想法!)
  • 尝试运行整个过程对于本地磁盘而不是网络磁盘上的文件,如果您的平均故障间隔时间是运行 10 次后,那么您可以在本地执行此加载 20-30 次,如果您没有收到错误,则可能 是网络错误

What error message (SSIS log / Eventvwr messages) do you get?

Similar to @Siva, I've not come across this, but some ideas you could use to try and diagnose. You may be doing some of these already, I've just written them down for completeness from my thought processes...

  • log all files processed. write a line to a log file/table pre-processing (each file), then post-process (each file). Keep the full path of each file. This is actually something we do as standard with our ETL implementations, as users are often coming back to us with questions about when/what has been loaded. This will allow you to see if files are actually being processed twice.
  • perhaps try moving each file after it is processed to a different directory. That will make it more difficult to have a file processed a second time and the problem may disappear. (If you are processing them from an area that is a "master" area (and so cant move them), consider copying the files to a "waiting" folder, then processing them and moving them to a "processed" folder)
  • @Siva's comment is interesting - look at the "traverse subfolders" check box.
  • check your eventvwr for odd network events, or application events (SQL Server restarting?)
  • use perf mon to see if there is anything odd happening in terms of network load on your server (a bit of a random idea!)
  • try running your whole process with files on a local disk instead of a network disk, if your mean time between failures is after running 10 times, then you could do this load locally 20-30 times and if you dont get an error it may be a network error
泅人 2024-12-02 17:59:51

没有任何帮助 - 我实现了以下解决方法:跟踪所有文件的 foreach 迭代器中的脚本任务。如果文件已加载,则会发出警告并且不会再次处理该文件。无论如何,似乎是一些与网络相关的问题......

nothing helped - I implemented following workaround: script task in the foreach iterator which tracks all files. if a file was alread loaded a warning is fired and the file is not processed again. anyway, seems to be some network related problem...

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文