修复 php 标记化模式

发布于 2024-11-28 07:04:12 字数 419 浏览 0 评论 0原文

我有一个 Perl 脚本，它标记一个字符串

@s=split /([^a-zA-Z \t\-\'\,\.]+)/, $_[0];  # tokenized with separators

，因此如果我有一个字符串 $s="The large [[bear]] isangered."

它将返回 array("The large", " [[", "bear", "]] ", "isanger", ".")

但是当我在 php 脚本上使用正则表达式模式时，它似乎不起作用。

$tokens = preg_split("/[^a-z \t\-\'\,\.]+/i", $s);

有人对这个问题有任何想法吗？

原文

i have a Perl script which tokenize a string

@s=split /([^a-zA-Z \t\-\'\,\.]+)/, $_[0];  # tokenized with separators

so if i have a string $s="The large [[bear]] is dangerous."

it will return array("The large", " [[", "bear", "]] ", "is dangerous", ".")

But the regex pattern doesn't seem to work when i used it on a php script.

$tokens = preg_split("/[^a-z \t\-\'\,\.]+/i", $s);

does anyone have any idea about the problem?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

一身仙ぐ女味 2024-12-05 07:04:12

让它运行（演示）：

$s="The large [[bear]] is dangerous.";
$pattern = '/([^a-zA-Z \t\-\\\'\,\.]+)/';    
$tokens = preg_split($pattern, $s, NULL, PREG_SPLIT_DELIM_CAPTURE);    
print_r($tokens);

输出：

Array
(
    [0] => The large
    [1] => [[
    [2] => bear
    [3] => ]]
    [4] =>  is dangerous.
)

详细信息：

模式必须正确表达为字符串在 PHP 中，请注意字符串转义序列。在单引号字符串中，\' 写为 \\\'。
如果您想要包含分隔符进行拆分，则需要使用 PREG_SPLIT_DELIM_CAPTURE 标志。

请参阅单引号字符串^文档和preg_split^文档

编辑： 要在标点符号处拆分，只需删除它们来自模式（演示）：

$pattern = '/([^a-zA-Z \t]+)/';

Got it to run (Demo):

$s="The large [[bear]] is dangerous.";
$pattern = '/([^a-zA-Z \t\-\\\'\,\.]+)/';    
$tokens = preg_split($pattern, $s, NULL, PREG_SPLIT_DELIM_CAPTURE);    
print_r($tokens);

Output:

Array
(
    [0] => The large
    [1] => [[
    [2] => bear
    [3] => ]]
    [4] =>  is dangerous.
)

Details:

The pattern must be properly expressed as a string in PHP, mind the string escape sequences. In a single quoted string \' is written as \\\'.
You want to split including the delimiters, you need to use the PREG_SPLIT_DELIM_CAPTURE flag.

See Single quoted Strings^Docs and preg_split^Docs

Edit: To split at the punctuation, just remove them from the pattern (Demo):