当前位置： 文江博客话题详情

perl如何正则表达式部分数据而不是整个字符串，然后打印出csv文件

发布于 2024-09-11 16:35:33 字数 5507 浏览 2 评论 0原文

我有一个工作 perl 脚本，可以获取我需要的数据并将其显示到 STDOUT，但现在我需要将其更改为生成数据文件（csv、制表符分隔符、任何分隔符文件）。 正则表达式正在过滤我需要的数据，但我不需要整个字符串，只需要输出的片段。我假设我需要将其存储在另一个变量中才能创建我的输出文件。

我需要一个很好的例子或建议来更改此代码。先感谢您。 :-)

这是我的代码：

#!/usr/bin/perl -w
# Usage: ./bakstatinfo.pl Jul 28 2010 /var/log/mybackup.log <server1> <server2>

use strict;
use warnings;

#This piece added to view the arguments passed in
$" = "][";
print "===================================================================================\n";
print "[@ARGV]\n";

#Declare Variables
my($mon,$day,$year,$file,$server) = @ARGV;
my $regex_flag = 0;                 

splice(@ARGV, 0, 4, ());            

foreach my $server ( @ARGV ) {      #foreach will take Xn of server entries and add to the loop
    print "===================================================================================\n";
    print "REPORTING SUMMARY for SERVER : $server\n";
    open(my $fh,"ssh $server cat $file |") or die "can't open log $server:$file: $!\n";
    while (my $line = <$fh>) {
        if ($line =~ m/.* $mon $day \d{2}:\d{2}:\d{2} $year:.*(ERROR:|backup-date=|backup-size=|backup-time=|backup-status)/) {
            print $line;
            $regex_flag=1; #Set to true
        }
    }
        if ($regex_flag==0) { 
           print "NOTHING TO REPORT FOR $server: $mon $day $year \n";
        }
    $regex_flag=0; 
    close($fh);
}

我正在使用的原始日志文件示例：（最近添加以提供更好的日志表示）

Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:ERROR: mybak-abc appears to be already running for this backupset
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:ERROR: If you are sure mybak-abc is not running, please remove the file /etc/mybak-abc/test202.bak_lvm/.mybak-abc.pid and restart mybak-abc
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:INFO: PHASE START: Cleanup
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:INFO: PHASE END: Cleanup
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:INFO: END OF BACKUP
Wed Jul 28 00:00:04 2010: db9.abc.bak:backup:INFO: START OF BACKUP
Wed Jul 28 00:00:04 2010: db9.abc.bak:backup:INFO: PHASE START: Initialization
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:WARNING: Binary logging is off.
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: License check successful
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: License check successful for lvm-snapshot.pl
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-set=db9.abc.bak
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-date=20100728000004
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: SQL-server-os=Linux/Unix
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-type=regular
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: host=db9.abc.bak.test.com
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-date-epoch=1280300404
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: retention-policy=3D
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: mybak-abc-version=ABC for SQL Enterprise Edition - version 3.1
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: SQL-version=5.1.32-test-SMP-log
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-directory=/home/backups/db9.abc.bak/20100728000004
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-level=0
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-mode=raw
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE END: Initialization
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE START: Running pre backup plugin
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE START: Flushing logs
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE END: Flushing logs
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE START: Creating snapshot based backup
Wed Jul 28 00:00:11 2010: db9.abc.bak:backup:INFO: Wed Jul 28 00:49:53 2010: test203.bak_lvm:backup:INFO: raw-databases-snapshot=test SQL sgl 
Wed Jul 28 00:49:53 2010: test203.bak_lvm:backup:INFO: PHASE END: Creating snapshot based backup
Wed Jul 28 00:49:53 2010: test203.bak_lvm:backup:INFO: PHASE START: Calculating backup size & checksums 
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: last-backup=/home/backups/test203.bak_lvm/20100726200004
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: PHASE END: Calculating backup size & checksums 
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: read-locks-time=00:00:05
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: flush-logs-time=00:00:00
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded

我现在的工作输出：

===================================================================================
[Jul][28][2010][/var/log/mybackup.log][server1]
===================================================================================
REPORTING SUMMARY for SERVER : server1
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded

我需要查看的输出将是这样的:(例如，数据文件以“;”分隔）

MyDate=Wed Jul 28;MyBackupSet= test203.bak_lvm;MyBackupSize=187.24 GB;MyBackupTime=04:49:51;MyBackupStat=Backup succeeded
原文

I have a working perl script that grabs the data I need and displays them to STDOUT, but now I need to change it to generate a data file (csv, tab dellimited, any delimiter file).
The regular expression is filtering the data that I need, but I don't want the entire string, just snippets of the output. I'm assuming I would need to store this in another variable to create my output file.

I need a good example of this or suggestions to alter this code. Thank you in advance. :-)

Here's my code:

#!/usr/bin/perl -w
# Usage: ./bakstatinfo.pl Jul 28 2010 /var/log/mybackup.log <server1> <server2>

use strict;
use warnings;

#This piece added to view the arguments passed in
$" = "][";
print "===================================================================================\n";
print "[@ARGV]\n";

#Declare Variables
my($mon,$day,$year,$file,$server) = @ARGV;
my $regex_flag = 0;                 

splice(@ARGV, 0, 4, ());            

foreach my $server ( @ARGV ) {      #foreach will take Xn of server entries and add to the loop
    print "===================================================================================\n";
    print "REPORTING SUMMARY for SERVER : $server\n";
    open(my $fh,"ssh $server cat $file |") or die "can't open log $server:$file: $!\n";
    while (my $line = <$fh>) {
        if ($line =~ m/.* $mon $day \d{2}:\d{2}:\d{2} $year:.*(ERROR:|backup-date=|backup-size=|backup-time=|backup-status)/) {
            print $line;
            $regex_flag=1; #Set to true
        }
    }
        if ($regex_flag==0) { 
           print "NOTHING TO REPORT FOR $server: $mon $day $year \n";
        }
    $regex_flag=0; 
    close($fh);
}

Sample raw log file I am using: (recently added to provide better representation of log)

Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:ERROR: mybak-abc appears to be already running for this backupset
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:ERROR: If you are sure mybak-abc is not running, please remove the file /etc/mybak-abc/test202.bak_lvm/.mybak-abc.pid and restart mybak-abc
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:INFO: PHASE START: Cleanup
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:INFO: PHASE END: Cleanup
Tue Jul 27 23:00:06 2010: test202.bak_lvm:backup:INFO: END OF BACKUP
Wed Jul 28 00:00:04 2010: db9.abc.bak:backup:INFO: START OF BACKUP
Wed Jul 28 00:00:04 2010: db9.abc.bak:backup:INFO: PHASE START: Initialization
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:WARNING: Binary logging is off.
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: License check successful
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: License check successful for lvm-snapshot.pl
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-set=db9.abc.bak
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-date=20100728000004
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: SQL-server-os=Linux/Unix
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-type=regular
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: host=db9.abc.bak.test.com
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-date-epoch=1280300404
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: retention-policy=3D
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: mybak-abc-version=ABC for SQL Enterprise Edition - version 3.1
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: SQL-version=5.1.32-test-SMP-log
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-directory=/home/backups/db9.abc.bak/20100728000004
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-level=0
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: backup-mode=raw
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE END: Initialization
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE START: Running pre backup plugin
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE START: Flushing logs
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE END: Flushing logs
Wed Jul 28 00:00:05 2010: db9.abc.bak:backup:INFO: PHASE START: Creating snapshot based backup
Wed Jul 28 00:00:11 2010: db9.abc.bak:backup:INFO: Wed Jul 28 00:49:53 2010: test203.bak_lvm:backup:INFO: raw-databases-snapshot=test SQL sgl 
Wed Jul 28 00:49:53 2010: test203.bak_lvm:backup:INFO: PHASE END: Creating snapshot based backup
Wed Jul 28 00:49:53 2010: test203.bak_lvm:backup:INFO: PHASE START: Calculating backup size & checksums 
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: last-backup=/home/backups/test203.bak_lvm/20100726200004
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: PHASE END: Calculating backup size & checksums 
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: read-locks-time=00:00:05
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: flush-logs-time=00:00:00
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded

My working output now:

===================================================================================
[Jul][28][2010][/var/log/mybackup.log][server1]
===================================================================================
REPORTING SUMMARY for SERVER : server1
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
Wed Jul 28 00:49:54 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded

The output I need to see would be something like this:(data file with separated by ';' for example)

MyDate=Wed Jul 28;MyBackupSet= test203.bak_lvm;MyBackupSize=187.24 GB;MyBackupTime=04:49:51;MyBackupStat=Backup succeeded

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（2）

一片旧的回忆 2024-09-18 16:35:33

使用“捕获括号”来标识您要处理的位。

   if ($line =~ m/(.* $mon $day) \d{2}:\d{2}:\d{2} $year:.*
                  (ERROR:|backup-date=|backup-size=|
                   backup-time=|backup-status)/x) {

您需要对第二组括号（围绕各个关键字开头的括号）进行一些操作。您可能需要在条件内将它们切成碎片。

将所有数据提取到变量中后，使用 Text::CSV 处理 CSV 输出（和输入）。

有无数的模块可以处理 HTML 或 XML（超过 2000 个，我想超过 3000 个，它们的名称中带有 HTML - 我昨天碰巧看到了）。其中许多内容并不适用，但 CPAN 是您的朋友。

回答评论提出的问题

我是否也将它们分成单独的变量？第一部分为我提供了我需要的日期/时间。然后下一个过滤器给我 1) 错误: 2)backup-date= 3)backup-size= ...etc.

或多或少。不幸的是，您没有显示一些有代表性的输入行，这意味着很难判断什么是最好的。然而，似乎有这样的方案：

while (my $line = <$fh>)
{
    chomp $line;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year:/)
    {
        my $date = $1;
        my %items = ();
        $line =~ s/.* $mon $day \d\d:\d\d:\d\d $year://;
        while ($line =~ m/(ERROR|backup-date|backup-size|
                           backup-time|backup-status)
                          [:=]([^:]+)/x)
        {
            my $key = $1;
            my $val = $2;
            $items{$key} = $val;
            $line =~ s/$key[:=]$val[:=]?//;
        }
        # The %items hash contains the split out information.
        # Now write the data for this line of the log file.
    }
}

很可能有更好的方法来处理修剪（但它是 Perl，所以 TMTOWTDI），但这里的基本思想是捕获感兴趣的行，然后逐步削减兴趣超出了行数，因此每次迭代行数都会变短（因此，最终终止内部 while 循环）。

请注意使用 /x 修饰符来允许更易读的正则表达式分割行（我编辑了原始答案版本也使用它）。我还允许“ERROR”后跟“=”或其他关键字后跟“:”；这样您似乎不太可能得到错误的匹配，并且它简化了正则表达式替换操作。初始模式匹配也不再需要存在其中一个小节。您必须自己判断这些微小的变化（可能会发现不合格的信息）是否重要。对于我的大多数目的来说，不匹配的可能性足够小，不会成为问题 - 但出于法律原因，您可能无法接受。

回答“答案”提出的问题时，

我制造了一些数据：

Wed Jul 30 00:49:51 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
Wed Jul 30 00:49:52 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
Wed Jul 30 00:49:53 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded
Wed Jul 30 00:49:51 2010: backup-size=417.32 GB:backup-time=04:49:51:backup-status=Backup succeeded

我将答案中的脚本进行了修改和检测，使其独立。
我还删除了对特定文件的依赖 - 它读取标准输入并写入标准输出。它使我的测试更容易 - 并且代码更灵活。

use strict;
use warnings;
use constant debug => 0;
my $mon = 'Jul';
my $day = 30;
my $year = 2010;

while (my $line = <>)
{
    chomp $line;
    print "Line: $line\n" if debug;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year:/) #Mon Jul 26 22:00:02 2010:
    {
        print "### Scan\n";
        my $date = $1;
        print "$date\n";
        my %items = ();
        $line =~ s/.* $mon $day \d\d:\d\d:\d\d $year://;
        print "Line: $line\n" if debug;
        while ($line =~ m/(ERROR|backup-date|backup-size|backup-time|backup-status)[:=]([^:]+)/)
        {
            my $key = $1;
            my $val = $2;
            $items{$key} = $val;
            $line =~ s/$key[:=]$val[:=]?//;
            print "$key=$val\n";
            print "Line: $line\n" if debug;
        }
        print "### Verify\n";
        for my $key (sort keys %items)
        {
            print "$key = $items{$key}\n";
        }
    }
}

我得到的输出是：

### Scan
Wed Jul 30
backup-size=417.32 GB
### Verify
backup-size = 417.32 GB
### Scan
Wed Jul 30
backup-time=04
### Verify
backup-time = 04
### Scan
Wed Jul 30
backup-status=Backup succeeded
### Verify
backup-status = Backup succeeded
### Scan
Wed Jul 30
backup-size=417.32 GB
backup-time=04
backup-status=Backup succeeded
### Verify
backup-size = 417.32 GB
backup-status = Backup succeeded
backup-time = 04

验证循环非常愉快地打印出“%items”哈希中的数据。将调试值设置为 1 而不是 0 时，我得到的输出是：

Line: Wed Jul 30 00:49:51 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
### Scan
Wed Jul 30
Line:  test203.bak_lvm:backup:INFO: backup-size=417.32 GB
backup-size=417.32 GB
Line:  test203.bak_lvm:backup:INFO: 
### Verify
backup-size = 417.32 GB
Line: Wed Jul 30 00:49:52 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
### Scan
Wed Jul 30
Line:  test203.bak_lvm:backup:INFO: backup-time=04:49:51
backup-time=04
Line:  test203.bak_lvm:backup:INFO: 49:51
### Verify
backup-time = 04
Line: Wed Jul 30 00:49:53 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded
### Scan
Wed Jul 30
Line:  test203.bak_lvm:backup:INFO: backup-status=Backup succeeded
backup-status=Backup succeeded
Line:  test203.bak_lvm:backup:INFO: 
### Verify
backup-status = Backup succeeded
Line: Wed Jul 30 00:49:51 2010: backup-size=417.32 GB:backup-time=04:49:51:backup-status=Backup succeeded
### Scan
Wed Jul 30
Line:  backup-size=417.32 GB:backup-time=04:49:51:backup-status=Backup succeeded
backup-size=417.32 GB
Line:  backup-time=04:49:51:backup-status=Backup succeeded
backup-time=04
Line:  49:51:backup-status=Backup succeeded
backup-status=Backup succeeded
Line:  49:51:
### Verify
backup-size = 417.32 GB
backup-status = Backup succeeded
backup-time = 04

替代操作删除了该行之前匹配的部分。有多种方法可以从您上次中断的位置继续进行匹配 - 请参阅 'perlre' 页面。

请注意，正则表达式被设计为在关键字后的“冒号或等于”之后的第一个冒号处停止。这意味着它会缩短备份时间。一个道德是“不要使用可能出现在数据中的分隔符”。另一个是“提供样本数据，以便人们可以更轻松地帮助您”。另一个是“尽可能提供完整但最少的工作脚本”。

处理示例数据

现在我们有了示例输入数据，我们可以看到您需要稍微不同的处理。此脚本：

use strict;
use warnings;
use constant debug => 0;
my $mon = 'Jul';
my $day = 28;
my $year = 2010;
my %items = ();

while (my $line = <>)
{
    chomp $line;
    print "Line: $line\n" if debug;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year: ([^:]+):backup:/) #Mon Jul 26 22:00:02 2010:
    {
        print "### Scan\n" if debug;
        my $date = $1;
        my $set = $2;
        print "$date ($set): " if debug;
        $items{$set}->{'a-logdate'} = $date;
        $items{$set}->{'a-dataset'} = $set;
        if ($line =~ m/(ERROR|backup-date|backup-size|backup-time|backup-status)[:=](.+)/)
        {
            my $key = $1;
            my $val = $2;
            $items{$set}->{$key} = $val;
            print "$key=$val\n" if debug;
        }
    }
}

print "### Verify\n";
for my $set (sort keys %items)
{
    print "Set: $set\n";
    my %info = %{$items{$set}};
    for my $key (sort keys %info)
    {
        printf "%s=%s;", $key, $info{$key};
    }
    print "\n";
}

在样本数据文件上生成此结果。

### Verify
Set: db9.abc.bak
a-dataset=db9.abc.bak;a-logdate=Wed Jul 28;backup-date=20100728000004;
Set: test203.bak_lvm
a-dataset=test203.bak_lvm;a-logdate=Wed Jul 28;backup-size=417.32 GB;backup-status=Backup succeeded;backup-time=04:49:51;

请注意，现在我们有了示例数据，我们可以看到每行只有一个键/值对，但每天有多个系统备份。因此，内部 while 循环变成了一个简单的 if。打印输出发生在最后。我正在使用“两层”哈希。 %items 包含每个数据集的条目；不过，该条目是对哈希值的引用。不一定适合新手玩，但它与之前的代码非常自然地融为一体。另请注意，此版本不会破解线路 - 没有必要，因为每行只有一批数据。

它可以改进吗——是的，毫无疑问。有效吗？是的，或多或少……它能被黑客改造吗？是的，它可以根据您的需要进行修改。

Use 'capturing parentheses' to identify the bits you want to deal with.

   if ($line =~ m/(.* $mon $day) \d{2}:\d{2}:\d{2} $year:.*
                  (ERROR:|backup-date=|backup-size=|
                   backup-time=|backup-status)/x) {

You will need to do some surgery on the second set of parentheses - those surrounding the start of the various keywords. You may have to chop those out in bits and pieces inside the condition.

When you have all the data extracted into variables, use Text::CSV to handle CSV output (and input).

There are a myriad modules to handle HTML or XML (over 2000, and I think over 3000, with HTML in their name - I happened to look yesterday). Many of those won't be applicable, but CPAN is your friend.

Answering questions posed by comments

Would I split them off into separate variables as well? The first part gives me the date/time that I need. The next filter then gives me 1) Error: 2)backup-date= 3)backup-size= ...etc.

More or less. Unfortunately, you don't show some representative input lines, which means it is hard to tell what might be best. However, it seems likely that a scheme such as:

while (my $line = <$fh>)
{
    chomp $line;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year:/)
    {
        my $date = $1;
        my %items = ();
        $line =~ s/.* $mon $day \d\d:\d\d:\d\d $year://;
        while ($line =~ m/(ERROR|backup-date|backup-size|
                           backup-time|backup-status)
                          [:=]([^:]+)/x)
        {
            my $key = $1;
            my $val = $2;
            $items{$key} = $val;
            $line =~ s/$key[:=]$val[:=]?//;
        }
        # The %items hash contains the split out information.
        # Now write the data for this line of the log file.
    }
}

There might well be better ways to handle the trimming (but it is Perl so TMTOWTDI), but the basic idea here is to catch the lines that are interesting, then progressively chop the bits of interest out of the line, so the line grows shorter on each iteration (therefore, eventually terminating the inner while loop).

Note the use of the /x modifier to allow for a more readable regex split over lines (I edited the original answer version to use that too). I've also allowed 'ERROR' to be followed by an '=' or the other keywords to be followed by ':'; it seems unlikely that you'd get false matches that way, and it simplifies the regex substitute operations. The initial pattern match no longer requires one of the subsections to be present, either. You must judge for yourself whether those small changes (which might pick up non-conforming information) matter or not. For most of my purposes, the chance of the mismatch is small enough not to be an issue - but for legal reasons, it might not be acceptable to you.

Answering questions posed by 'answer'

I manufactured some data:

Wed Jul 30 00:49:51 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
Wed Jul 30 00:49:52 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
Wed Jul 30 00:49:53 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded
Wed Jul 30 00:49:51 2010: backup-size=417.32 GB:backup-time=04:49:51:backup-status=Backup succeeded

I took the script in the answer and hacked and instrumented it - making it standalone.
I also removed the dependency on specific files - it reads standard input and writes to standard output. It makes my testing easier - and the code more flexible.

use strict;
use warnings;
use constant debug => 0;
my $mon = 'Jul';
my $day = 30;
my $year = 2010;

while (my $line = <>)
{
    chomp $line;
    print "Line: $line\n" if debug;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year:/) #Mon Jul 26 22:00:02 2010:
    {
        print "### Scan\n";
        my $date = $1;
        print "$date\n";
        my %items = ();
        $line =~ s/.* $mon $day \d\d:\d\d:\d\d $year://;
        print "Line: $line\n" if debug;
        while ($line =~ m/(ERROR|backup-date|backup-size|backup-time|backup-status)[:=]([^:]+)/)
        {
            my $key = $1;
            my $val = $2;
            $items{$key} = $val;
            $line =~ s/$key[:=]$val[:=]?//;
            print "$key=$val\n";
            print "Line: $line\n" if debug;
        }
        print "### Verify\n";
        for my $key (sort keys %items)
        {
            print "$key = $items{$key}\n";
        }
    }
}

The output I get is:

### Scan
Wed Jul 30
backup-size=417.32 GB
### Verify
backup-size = 417.32 GB
### Scan
Wed Jul 30
backup-time=04
### Verify
backup-time = 04
### Scan
Wed Jul 30
backup-status=Backup succeeded
### Verify
backup-status = Backup succeeded
### Scan
Wed Jul 30
backup-size=417.32 GB
backup-time=04
backup-status=Backup succeeded
### Verify
backup-size = 417.32 GB
backup-status = Backup succeeded
backup-time = 04

The verify loop prints out the data from the '%items' hash quite happily. With the debug value set to 1 instead of 0, the output I get is:

Line: Wed Jul 30 00:49:51 2010: test203.bak_lvm:backup:INFO: backup-size=417.32 GB
### Scan
Wed Jul 30
Line:  test203.bak_lvm:backup:INFO: backup-size=417.32 GB
backup-size=417.32 GB
Line:  test203.bak_lvm:backup:INFO: 
### Verify
backup-size = 417.32 GB
Line: Wed Jul 30 00:49:52 2010: test203.bak_lvm:backup:INFO: backup-time=04:49:51
### Scan
Wed Jul 30
Line:  test203.bak_lvm:backup:INFO: backup-time=04:49:51
backup-time=04
Line:  test203.bak_lvm:backup:INFO: 49:51
### Verify
backup-time = 04
Line: Wed Jul 30 00:49:53 2010: test203.bak_lvm:backup:INFO: backup-status=Backup succeeded
### Scan
Wed Jul 30
Line:  test203.bak_lvm:backup:INFO: backup-status=Backup succeeded
backup-status=Backup succeeded
Line:  test203.bak_lvm:backup:INFO: 
### Verify
backup-status = Backup succeeded
Line: Wed Jul 30 00:49:51 2010: backup-size=417.32 GB:backup-time=04:49:51:backup-status=Backup succeeded
### Scan
Wed Jul 30
Line:  backup-size=417.32 GB:backup-time=04:49:51:backup-status=Backup succeeded
backup-size=417.32 GB
Line:  backup-time=04:49:51:backup-status=Backup succeeded
backup-time=04
Line:  49:51:backup-status=Backup succeeded
backup-status=Backup succeeded
Line:  49:51:
### Verify
backup-size = 417.32 GB
backup-status = Backup succeeded
backup-time = 04

The substitute operations delete the previously matched part of the line. There are ways of continuing a match where you left off - see \G at the 'perlre' page.

Note that the regex is crafted to stop at the first colon after the 'colon or equals' after the keyword. That means it truncates the backup time. One moral is "do not use a separator that can appear in the data". Another is "provide sample data so people can help you more easily". Another is "provide complete but minimal working scripts where possible".

Processing the sample data

Now that we have the sample input data, we can see that you need slightly different processing. This script:

use strict;
use warnings;
use constant debug => 0;
my $mon = 'Jul';
my $day = 28;
my $year = 2010;
my %items = ();

while (my $line = <>)
{
    chomp $line;
    print "Line: $line\n" if debug;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year: ([^:]+):backup:/) #Mon Jul 26 22:00:02 2010:
    {
        print "### Scan\n" if debug;
        my $date = $1;
        my $set = $2;
        print "$date ($set): " if debug;
        $items{$set}->{'a-logdate'} = $date;
        $items{$set}->{'a-dataset'} = $set;
        if ($line =~ m/(ERROR|backup-date|backup-size|backup-time|backup-status)[:=](.+)/)
        {
            my $key = $1;
            my $val = $2;
            $items{$set}->{$key} = $val;
            print "$key=$val\n" if debug;
        }
    }
}

print "### Verify\n";
for my $set (sort keys %items)
{
    print "Set: $set\n";
    my %info = %{$items{$set}};
    for my $key (sort keys %info)
    {
        printf "%s=%s;", $key, $info{$key};
    }
    print "\n";
}

produces this result on the sample data file.

### Verify
Set: db9.abc.bak
a-dataset=db9.abc.bak;a-logdate=Wed Jul 28;backup-date=20100728000004;
Set: test203.bak_lvm
a-dataset=test203.bak_lvm;a-logdate=Wed Jul 28;backup-size=417.32 GB;backup-status=Backup succeeded;backup-time=04:49:51;

Note that now we have sample data, we can see that there is only one key/value pair per line, but there are multiple systems backed up per day. So, the inner while loop becomes a simple if. The printing out occurs at the end. And I'm using a 'two-tier' hash. The %items contains an entry for each data set; the entry, though, is a reference to a hash. Not necessarily something for novices to play with, but it fell into place very naturally with the previous code. Note, too, that this version doesn't hack the line - there's no need since there's only one lot of data per line.

Can it be improved - yes, undoubtedly. Does it work? Yes, more or less... Can it be hacked into shape? Yes, it can be hacked to work as you need.

回复 原文
驱逐舰岛风号 2024-09-18 16:35:33

@Jonathan-我在 while 循环中写出了文本文件。似乎有效。正如您在评论中所建议的那样，我尝试在第二个 while 循环之后执行此操作。我不知道为什么它不起作用。

open (my $MYDATAFILE, ">/home/test/myout.txt") || die "cannot append $!";
open(my $fh,"ssh $server cat $file |") or die "can't open log $server:$file: $!\n";
while (my $line = <$fh>)
{
    chomp $line;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year:/) #Mon Jul 26 22:00:02 2010:
    {
        my $date = $1;
        #print $date;
        my %items = ();
        $line =~ s/.* $mon $day \d\d:\d\d:\d\d $year://;
        while ($line =~ m/(ERROR|backup-date|backup-size|backup-time|backup-status)[:=]([^:]+)/)
        {
            my $key = $1;
            my $val = $2;
            $items{$key} = $val;
            $line =~ s/$key[:=]$val[:=]?//;
            #print "[$key]";
            #print "[$val]";
            print $MYDATAFILE "$key=$val";
        }
        # The %items hash contains the split out information.
        # Now write the data for this line of the log file.
    }
}

@Jonathan- I wrote out the text file within the while loop. It seems to work. I tried doing it after the second while loop as you suggested in your comment. I'm not sure why it didn't work.

open (my $MYDATAFILE, ">/home/test/myout.txt") || die "cannot append $!";
open(my $fh,"ssh $server cat $file |") or die "can't open log $server:$file: $!\n";
while (my $line = <$fh>)
{
    chomp $line;
    if ($line =~ m/(.* $mon $day) \d\d:\d\d:\d\d $year:/) #Mon Jul 26 22:00:02 2010:
    {
        my $date = $1;
        #print $date;
        my %items = ();
        $line =~ s/.* $mon $day \d\d:\d\d:\d\d $year://;
        while ($line =~ m/(ERROR|backup-date|backup-size|backup-time|backup-status)[:=]([^:]+)/)
        {
            my $key = $1;
            my $val = $2;
            $items{$key} = $val;
            $line =~ s/$key[:=]$val[:=]?//;
            #print "[$key]";
            #print "[$val]";
            print $MYDATAFILE "$key=$val";
        }
        # The %items hash contains the split out information.
        # Now write the data for this line of the log file.
    }
}
回复 原文
~没有更多了~

关于作者

街角迷惘

暂无简介

0 文章
0 评论
23 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

内心激荡

文章 0 评论 0

JSmiles

文章 0 评论 0

赏烟花じ飞满天

文章 0 评论 0

左秋

文章 0 评论 0

迪街小绵羊

文章 0 评论 0

瞳孔里扚悲伤

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文