linux commands 常用命令介绍
查看当前目录下的各个子目录大小
# https://superuser.com/questions/554319/display-each-sub-directory-size-in-a-list-format-using-one-line-command-in-bash [root@tencent-beta18 logs]# du -h --max-depth=1 98M ./bjca-api-statistics-tasks-app 173M ./bjca-app-api-manager-server 512K ./apollo 1.4M ./bjca-app-msgcenter-server 23M ./arthas 44K ./bjca-app-config-cases-server 2.7M ./bjca-bedrock-oss-app 184K ./bjca-auth-code-server-app 84K ./bjca-api-usercenter-server-app 44K ./elastic-job 4.0K ./arthas-cache 145M ./bjca-app-usercenter-tasks 324K ./bjca-api-statistics-server-app 63G ./bjca-app-usercenter 1.8G ./bjca-app-metrics-ump-server 13M ./bjca-app-elastic-server 8.0K ./simple-jar-demo-server 28K ./bjca-auth-code-tasks-app 12M ./bjca-app-xxx-server 65G .
ps aux
ps aux output meaning
➜ rig git:(master) ps aux | head -5 USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND bingoobjca 1522 13.0 1.4 5568692 240992 ?? S 一09上午 22:33.13 /Applications/iTerm.app/Contents/MacOS/iTerm2 bingoobjca 10559 4.1 0.0 4371836 4960 s002 S 10:13上午 0:03.01 -zsh bingoobjca 1361 3.8 1.3 6239448 218640 ?? S 一09上午 25:19.07 /Applications/QQ.app/Contents/MacOS/QQ bingoobjca 78181 2.8 9.7 179434016 1629116 ?? S 四10上午 58:29.96 /Applications/GoLand.app/Contents/MacOS/goland
- USER = user owning the process 拥有该流程的用户，摘
- PID = process ID of the process 流程的进程ID
- %CPU = It is the CPU time used divided by the time the process has been running. 使用的CPU时间除以进程运行的时间。
- %MEM = ratio of the process’s resident set size to the physical memory on the machine 进程驻留设置大小与计算机上物理内存的比率
- VSZ = virtual memory usage of entire process (in KiB) 整个过程的虚拟内存使用量（以KiB为单位）
- RSS = resident set size, the non-swapped physical memory that a task has used (in KiB) 驻留集大小，任务使用的非交换物理内存（以KiB为单位）
- TTY = controlling tty (terminal) 控制tty（终端）
- STAT = multi-character process state 多字符流程状态
- START = starting time or date of the process 流程的开始时间或日期
- TIME = cumulative CPU time 累计CPU时间
- COMMAND = command with all its arguments 带有所有参数的命令
See the ps man page for more info.
Difference between VSZ vs RSS memory usage
VSZ - Virtual Set Size
The Virtual Set Size is a memory size assigned to a process ( program ) during the initial execution. The Virtual Set Size memory is simply a number of how much memory a process has available for its execution.
RSS - Resident Set Size
As oppose to VSZ ( Virtual Set Size ), RSS is a memory currently used by a process. This is a actual number in kilobytes of how much RAM the current process is using.
简单的说 VSZ 代表的是虚拟记忆体的使用量，RSS代表实体记忆体的使用量。但是这里RSS的值，是包括shared memory的。Linux会把一些share library(lib*.so)载入记忆体中，而不同的 process 会共享部份的 share library，也就是说这些share library只被Linux在载入一次，而process可以从重复引用。
ps aux | awk '$2 == pid { print $0 }
➜ rig git:(master) ps aux | awk '$2 == 1522 { print $3,$4,$6,$0 }'
7.9 1.4 240720 bingoobjca 1522 7.9 1.4 5567532 240720 ?? S 一09上午 22:51.44 /Applications/iTerm.app/Contents/MacOS/iTerm2按 RSS 倒排序 ps aux | sort -rn -k 6 |head -n 5
➜ rig git:(master) ps aux | sort -rn -k 6 |head -n 5 bingoobjca 78181 3.0 9.7 179445460 1631240 ?? S 四10上午 59:12.60 /Applications/GoLand.app/Contents/MacOS/goland bingoobjca 510 0.0 2.7 6215516 458620 ?? S 一09上午 59:10.09 /Applications/Google Chrome.app/Contents/MacOS/Google Chrome bingoobjca 48414 1.1 2.1 4814908 359972 ?? S 三11上午 13:47.57 /Applications/VirtualBox.app/Contents/MacOS/VBoxHeadless --comment centos-kerb_default_1560232592051_15800 --startvm 82973c29-834c-497e-94f9-217e2eb7d49b --vrde config bingoobjca 1522 5.4 1.7 5706412 285892 ?? S 一09上午 22:56.76 /Applications/iTerm.app/Contents/MacOS/iTerm2 bingoobjca 75281 0.0 1.7 5710188 285860 ?? S 四08上午 4:20.78 /Applications/WeChat.app/Contents/MacOS/WeChat
The TLDR pages are a community effort to simplify the beloved man pages with practical examples. tldr tar
# bingoo @ 192 in ~/github/dog on git:master o [11:57:37]
$ tldr tar
tar
Archiving utility.
Often combined with a compression method, such as gzip or bzip.
More information: <https://www.gnu.org/software/tar>.
- Create an archive from files:
tar -cf target.tar file1 file2 file3
- Create a gzipped archive:
tar -czf target.tar.gz file1 file2 file3
- Extract an archive in a target directory:
tar -xf source.tar -C directory
- Extract a gzipped archive in the current directory:
tar -xzf source.tar.gz
- Extract a bzipped archive in the current directory:
tar -xjf source.tar.bz2
- Create a compressed archive, using archive suffix to determine the compression program:
tar -caf target.tar.xz file1 file2 file3
- List the contents of a tar file:
tar -tvf source.tar
- Extract files matching a pattern:
tar -xf source.tar --wildcards "*.html"list files order by size desc:
[betaoper@beta-hetong default]$ find . -type f -exec du -h {} + | sort -r -h |head -n 10
16M ./2019-07-09/1148515944367390722Convert.PDF
12M ./2019-06-28/1144528088150097921.PDF
7.9M ./2019-07-02/1146051722958139394Convert.PDF
7.2M ./2019-06-28/1144476741342109698.PDF
5.2M ./2019-07-01/1145605641686265857.PDF
4.6M ./2019-07-02/1146051549095849986.PDF
4.6M ./2019-07-02/1146050677066493953.PDF
4.1M ./2019-07-02/1146057879357550594.DOCX
4.1M ./2019-07-02/1146053558574313475.DOCX
3.5M ./2019-07-04/1146604946106630145.DOCX指定用户名，进行 svn 操作: svn co URL --username xxx --password xxx
查看 linux 中某个端口（port）是否被占用
lsof lsof -i:3306netstat -anp|grep 3306
查看局域网 IP 列表
方法1， nmap
Install nmap, sudo apt-get install nmap then nmap -sP 192.168.1.* or more commonly nmap -sn 192.168.1.0/24 will scan the entire .1 to .254 range
This does a simple ping scan in the entire subnet to see which hosts are online.
$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255
ether 52:54:00:ef:16:bd txqueuelen 1000 (Ethernet)
RX packets 8929283010 bytes 3287811980423 (2.9 TiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8071523788 bytes 3525672558442 (3.2 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1 (Local Loopback)
RX packets 2761872043 bytes 1133502059424 (1.0 TiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2761872043 bytes 1133502059424 (1.0 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
$ nmap -sP 192.168.1.*
Starting Nmap 6.40 ( http://nmap.org ) at 2019-11-26 11:01 CST
Nmap scan report for hadoop3 (192.168.1.3)
Host is up (0.00030s latency).
Nmap scan report for 192.168.1.5
Host is up (0.00080s latency).
Nmap scan report for hadoop1 (192.168.1.7)
Host is up (0.000088s latency).
Nmap scan report for 192.168.1.8
Host is up (0.00044s latency).
Nmap scan report for 192.168.1.9
Host is up (0.00042s latency).
Nmap scan report for hadoop2 (192.168.1.11)
Host is up (0.00060s latency).
Nmap scan report for hadoop0 (192.168.1.14)
Host is up (0.00031s latency).
Nmap scan report for 192.168.1.16
Host is up (0.00055s latency).
Nmap done: 256 IP addresses (8 hosts up) scanned in 2.41 seconds方法2 arp
Try following steps:
- Type ipconfig (or ifconfig on Linux) at command prompt. This will give you the IP address of your own machine. For example, your machine's IP address is 192.168.1.6. So your broadcast IP address is 192.168.1.255.
- Ping your broadcast IP address ping 192.168.1.255 (may require -b on Linux)
- Now type arp -a. You will get the list of all IP addresses on your segment.
$ arp -a ? (169.254.0.4) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (169.254.0.74) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (169.254.1.12) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (172.18.0.6) at 02:42:ac:12:00:06 [ether] on br-8983f91a1c88 ? (172.17.0.2) at 02:42:ac:11:00:02 [ether] on docker0 ? (169.254.1.5) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (169.254.0.23) at fe:ee:ec:9b:c5:79 [ether] on eth0 hadoop0 (192.168.1.14) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (169.254.1.17) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (172.18.0.8) at 02:42:ac:12:00:08 [ether] on br-8983f91a1c88 ? (172.18.0.3) at <incomplete> on br-8983f91a1c88 ? (169.254.0.15) at fe:ee:ec:9b:c5:79 [ether] on eth0 gateway (192.168.1.1) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (172.18.0.10) at 02:42:ac:12:00:0a [ether] on br-8983f91a1c88 ? (169.254.1.2) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (169.254.0.2) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (192.168.1.10) at fe:ee:ec:9b:c5:79 [ether] on eth0 hadoop3 (192.168.1.3) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (169.254.0.55) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (169.254.0.3) at fe:ee:ec:9b:c5:79 [ether] on eth0 hadoop2 (192.168.1.11) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (192.168.1.118) at fe:ee:ec:9b:c5:79 [ether] on eth0 ? (172.18.0.7) at <incomplete> on br-8983f91a1c88
grep a file, but show several surrounding lines?
For BSD or GNU grep you can use -B num to set how many lines before the match and -A num for the number of lines after the match.
grep -B 3 -A 2 foo README.txt
If you want the same number of lines before and after you can use -C num.
grep -C 3 foo README.txt
This will show 3 lines before and 3 lines after.
-A and -B will work, as will -C n (for n lines of context), or just -n (for n lines of context... as long as n is 1 to 9).
To build on @Livven's answer, here are some useful combinations.
1. variable substitution, leading tab retained, overwrite file, echo to stdout
tee /path/to/file <<EOF
${variable}
EOF2. no variable substitution, leading tab retained, overwrite file, echo to stdout
tee /path/to/file <<'EOF'
${variable}
EOF3. variable substitution, leading tab removed, overwrite file, echo to stdout
tee /path/to/file <<-EOF
${variable}
EOF4. variable substitution, leading tab retained, append to file, echo to stdout
tee -a /path/to/file <<EOF
${variable}
EOF5. variable substitution, leading tab retained, overwrite file, no echo to stdout
tee /path/to/file <<EOF >/dev/null
${variable}
EOF6. the above can be combined with sudo as well
sudo -u USER tee /path/to/file <<EOF
${variable}
EOF
#!/bin/bash
# bccrack.sh
cd "/Applications/Beyond Compare.app/Contents/MacOS"
if [[ -f BCompare.real ]]; then
exit 0
fi
mv BCompare BCompare.real
cat > BCompare <<'EOF'
#!/bin/bash
rm "/Users/$(whoami)/Library/Application Support/Beyond Compare/registry.dat"
"`dirname "$0"`"/BCompare.real $@
EOF
chmod +x BCompare如何使用 Journalctl 查看和操作 Systemd 日志
journalctl -u service-name.service-u flag is short for --unitjournalctl -u service-name.service -bto see only log messages for the current boot, -b is short for --bootjournalctl -u service-nameFor things named .service, you can actually just usejournalctl --since "2018-08-30 14:10:10" --until "2018-09-02 12:05:50"journalctl --since "-1h15min"journalctl --since "2015-01-10" --until "2015-01-11 03:00"journalctl --since yesterdayjournalctl -n显示最近的10个条目journalctl -n 20显示最近的20个条目journalctl _PID=8088 -o jsonjournalctl --since 09:00 --until "1 hour ago"journalctl -fview a live log of new messages as they are collected
Thanks:
Linux 网卡，防火墙，转发
Manipulating network interfaces, firewalling, and forwarding from Go.
Manipulating network interfaces, firewalling, and forwarding from Go.
The last few years of development on Linux have been exciting on the network front:
- NFTables - A high performance replacement for IPTables, NFTables provides a sophisticated (bytecode-based) rules engine, and the ability to make atomic rule changes (something that IPTables sorely lacked).
- Netlink - Netlink is a generic interface to features inside the kernel, but ends up being prevalent (and very convenient) for manipulating the network stack.
- Network Namespaces - Okay this isn't new, but network namespaces in-part enabled the containerization of server workloads, network sandboxing, and are just awesome in general.
Best of all, we can automate everything using Go.
The Basics: Configuring your interfaces
A pre-requisite to any funky networking is bringing the interface up. This is trivial from Go, thanks to the rtnetlink API, codified by vishvananda's fabulous netlink package.
// Get a structure describing the network interface.
localInterface, err := netlink.LinkByName("eth0")
if err != nil {
// handle error
}
// Give the interface an address of 192.168.1.1, on a
// network with a 255.255.255.0 mask.
ipConfig := &netlink.Addr{IPNet: &net.IPNet{
IP: net.ParseIP("192.168.1.1"),
Mask: net.CIDRMask(24, 32)
}}
if err = netlink.AddrAdd(localInterface, ipConfig); err != nil {
// handle error
}
// Setup the default route, so traffic that doesn't hit
// 192.168.1.(1-255) can be routed.
if err = netlink.RouteAdd(&netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
LinkIndex: localInterface.Attrs().Index,
Dst: &net.IPNet{IP: gatewayIP, Mask: net.CIDRMask(32, 32)},
}); err != nil {
// handle error
}
// Lastly, bring up the interface.
if err = netlink.LinkSetUp(localInterface); err != nil {
// handle error
}Under the hood, the following is occurring:
- A socket of family AF_NETLINK is opened.
- Using the NETLINK_ROUTE protocol and RTM_GETLINK / RTM_SETLINK messages, the package queries and sets link information.
- Lastly, the RTM_NEWROUTE message is used to set the default gateway.
If you want to read about the wire encoding for this & other netlink interfaces, have a look at Matt Layher's posts on the subject.
Using NFTables for packet filtering
NFTables, the successor to IPTables, is a highly-configurable rules engine for processing packets. It is configured via a netlink interface, much like the example above.
A primer on NFTables
nftables 是新式的数据包过滤框架，从 Linux 内核 3.13 版本开始可用，旨在替代现用的 iptables 框架。Nftables HOWTO 中文翻译，更多 nftables 相比 iptables 到底改变了什么
First, some terminology:
- Tables - Tables in NFTables are simply a container. A table can be associated with a single address family (ie: IPv4, IPv6, inet. I use inet for most of my tables as it works on both IPv4 & IPv6 traffic). Rules are contained within chains, and chains are contained within a Table.
- Chains - An NFTables chain is a hook into Linux's packet processing pipeline. You attach your rules to a specific chain, which will cause the rules to be evaluated whenever a packet hits a certain point in Linux's pipeline. For instance, if you attach rules to a chain of type prerouting, the rules will be evaluated before Linux looks at routes & makes a routing decision.
When you specify a chain, you need to provide a type. I don't know exactly what this means, but its pretty logical. If you want to filter packets, use the type filter. If you want to re-write routing information (IPTables calls this mangling) use type route. If you are using NAT features, use type NAT. - Rules - Rules are where your filtering/forwarding/whatever logic go. We talk about building these in the next section.
NFTables packet pipeline (abridged version)
The triangles represent chains. Conceptually, it makes sense to think of them as hooks.
Putting it all together
So, if you want to filter out certain IPv4 traffic from the internet from everything, you would put your rules in a prerouting chain with hook type filter, in a table with a name of your choosing. The table would have an address family of IPv4.
Hopefully this section gives you enough information to fill in the opaque-sounding fields. The rest is pretty straightforward.
Setting up NFTables from Go
I've talked for long enough, lets get into some code shall we? We use the nftables package by stapelberg, which is shaping up to be the canonical Go package for nftables (I'm biased thou, I'm sending PRs there too).
c := &nftables.Conn{}
// Lets create our table first.
myTable := c.AddTable(&nftables.Table{
Family: nftables.TableFamilyIPv4,
Name: "myFilter",
})
// Now, we create a chain which we will add our filter
// rules to.
myChain := c.AddChain(&nftables.Chain{
Name: "myChain",
Table: myTable,
Type: nftables.ChainTypeFilter,
Hooknum: nftables.ChainHookInput,
Priority: nftables.ChainPriorityFilter,
})
// Lets add a rule to the chain that loads the source
// address, and compares it to a hardcoded IP.
c.AddRule(&nftables.Rule{
Table: myTable,
Chain: myChain,
Exprs: []expr.Any{
// payload load 4b @ network header + 12 => reg 1
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: 12,
Len: 4,
},
// cmp eq reg 1 0x0245a8c0
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: net.ParseIP("192.168.1.53").To4(),
},
// [ immediate reg 0 drop ]
&expr.Verdict{
Kind: expr.VerdictDrop,
},
},
})
// Apply the above (commands are queued till a call to Flush())
if err := c.Flush(); err != nil {
// handle error
}Lets break this down.
First, we construct a literal nftables.Conn. This connection contains all the state of our table.
Next, we define and create our table. Aside from naming, the only field we have to choose here is the family. As this example highlights dropping traffic based on an IPv4 address, it makes sense for it to be in the IPv4 family.
Creating the chain is where the fields get confusing. We create a chain in the table we previously created, with the following fields:
Filtertype - Because we want to filter packetsInputchain (hook) - Because we want to drop packets before they enter the system (but not interfere with forwarded traffic)Filterpriority - I havent talked about priority yet, because you should seldom need to change it. Priorities other than Filter allows you to make your rules run before/after internal nftables operations. See the nftables wiki for more details.
Next, we add a rule to drop packets where the source IP is 192.168.1.53. We break down the composition of rules in the next section.
Lastly, we call Flush() to apply everything in one go (The table, chain, nor rules are not created till you call Flush). While this may seem like a confusing API design decision, batching NFTables changes into atomic transactions allows you to avoid strange behaviour, when packets are recieved between two NFTables changes.
Congrats! You've just applied your first NFTables rule to the system!
NFTables Rules
I've avoided writing about these till now as they are a little confusing at first. That said, the rule engine is one of the most powerful features NFTables has on offer.
An NFTables rule is composed of a series of expressions, which are evaluated from beginning to end. If the expression checks some condition, and the condition evaluates to false, the remainder of the expressions are not evaluated (and hence any actions in later expressions are not applied).
Other NFTables rules perform some action. For instance expr.Drop instructs NFTables to drop the packet if it is evaluated (conversely, expr.Accept tells NFTables to accept the packet). There are also actions that increment counters, actions that mangle packets, actions that mark the connection or NAT it, and many more. Lets break down the example we saw before:
// payload load 4b @ network header + 12 => reg 1
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: 12,
Len: 4,
},This expression loads 4 bytes (the size of an IPv4 address) from offset 12 of the packet's network header. If we look at the wikipedia page for an IPv4 packet, we can see that offset 12 is the source IP address. Neat!
There are 20, 32bit registers you can use to store state as your rule is evaluated. Register 0 specifies the verdict (ie: accept/drop packet) code, so don't use it unless you know what you are doing.
// cmp eq reg 1 0x0245a8c0
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: net.ParseIP("192.168.1.53").To4(),
},The next blob of code compares the contents of register 1, permitting the rule to continue evaluation if the register contents match (CmpOpEq) the data (192.168.1.53). This has the effect of aborting rule evaluation if the source IP of the packet is not 192.168.1.53.
The last expression expr.Verdict simply drops the packet if the rule gets that far in evaluation (internally, this sets register 0). Since only packets with a source IP of 192.168.1.53 get this far, the expressions in this rule collectively add up to drop packets from 192.168.1.53.
Future posts
There's heaps more to cover but this post is getting long.
Building advanced NFTables rules- I'll explain how to come up with more complex rules, and provide a load of copypasta for the common use cases.NFTables counters & sets- NFTables introduced the ability to use in-kernel lookup datastructures called sets, which makes most rules drastically simpler, & better support the common use-cases (blocking/allowing/forwarding sets of IPs/ports). Counters also let you track the throughput of your rules, both in terms of the number of packets and the number of bytes.Filtering process traffic with net namespaces- In this post I'll go over network namespaces, how to use them, and walk through a network filter for select processes.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论