初访iptables及openstack虚拟网络
背景
上篇文章中，跨命名空间启动tftp服务后，需要进行验证，由于环境所限，使用kvm虚拟机进行验证。
kvm是openstack原生支持的虚拟化，在计算节点上通过iptables增加了许多规则以保证网络安全。
从而导致由虚拟机访问网络节点tftp服务总是失败，因此需要调查是哪条规则使然。
通过抓包tap卡发现，从虚拟机发送到网络节点的包可以通过，但由网络节点返回的包到达不了虚拟机，可以确认是网络包通过计算节点上linuxbridge时被丢弃，因此着手查iptables规则
概念
TAP/TUN是Linux内核实现的一对虚拟网络设备，TAP工作在二层，TUN工作在三层，Linux内核通过TAP/TUN设备向绑定该设备的用户空间程序发送数据，反之，用户空间程序也可以像操作硬件网络设备那样，通过TAP/TUN设备发送数据。
基于TAP驱动，即可以实现虚拟网卡的功能，虚拟机的每个vNIC都与hypervisor中的一个TAP设备相连。当一个TAP设备被创建时，在Linux设备文件目录下生会生成一个对应的字符设备文件，用户程序可以打开普通文件一样打开这个文件进行读写。当对这个TAP设备文件执行write()操作时，对于Linux网络子系统来说，相对于TAP设备收到了数据，并请求内核接受他，Linux内核收到此数据后将根据网络配置进行后续处理，处理过程类似于普通的物理网卡从外界收到数据。（VM向外发数据）。
当用户程序执行read()请求时，相对于向内核查询TAP设备上是否有数据需要被发送，有的话则取出到用户程序里，从而完成TAP设备发送数据的功能。（VM接收数据）。
在这个过程中，TAP设备可以被当做本机的一个网卡，而操作TAP设备应用程序相对于另外一台VM，它通过read/write系统调用，和本机进行网络通信。
VETH设备总是成对出现，送到一端请求发送的数据总是从另一端以请求接收的形式出现。创建并配置正确后，向其一端输入数据，VETH会改变数据的方向并将其送入内核网络子系统，完成数据的注入，而在另一端则能读到此数据。
tun：点对点的设备，tun设备模拟网络层设备，处理三层报文，如IP报文。tun设备完全不需要物理地址的。它收到和发出的包不需要arp，也不需要有数据链路层的头。
tap：是一个普通的以太网设备，tap设备模拟链路层设备，处理二层报文，比如以太网帧。tap设备需要有完整的物理地址和完整的以太网帧
TUN用于路由，而TAP用于创建网桥。
引用：https://blog.csdn.net/chengqiuming/article/details/79840092?utm_source=blogxgwz8
iptables概念
openstack在创建kvm虚拟机过程中，创建了很多iptables的自定义链，以保证网络安全：
# iptables -L -vn --line-number
Chain INPUT (policy ACCEPT 125K packets, 18M bytes)
num pkts bytes target prot opt in out source destination
1 550K 80M neutron-openvswi-INPUT all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 1574 111K neutron-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
2 1574 111K neutron-openvswi-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 131K packets, 19M bytes)
num pkts bytes target prot opt in out source destination
1 556K 78M neutron-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
2 556K 78M neutron-openvswi-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0
Chain neutron-filter-top (2 references)
num pkts bytes target prot opt in out source destination
1 558K 79M neutron-openvswi-local all -- * * 0.0.0.0/0 0.0.0.0/0
Chain neutron-openvswi-FORWARD (1 references)
num pkts bytes target prot opt in out source destination
1 11 1904 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap2c6951d5-e6 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
2 0 0 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap2c6951d5-e6 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
3 865 37140 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap6746373c-01 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
4 179 12312 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap6746373c-01 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
Chain neutron-openvswi-INPUT (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 neutron-openvswi-o2c6951d5-e all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap2c6951d5-e6 --physdev-is-bridged /* Direct incoming traffic from VM to the security group chain. */
2 0 0 neutron-openvswi-o6746373c-0 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap6746373c-01 --physdev-is-bridged /* Direct incoming traffic from VM to the security group chain. */
Chain neutron-openvswi-OUTPUT (1 references)
num pkts bytes target prot opt in out source destination
Chain neutron-openvswi-i2c6951d5-e (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
2 0 0 RETURN udp -- * * 14.52.45.2 0.0.0.0/0 udp spt:67 udp dpt:68
3 6 312 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv4b3a1256e-07fe-4e6b-bfc5- src
4 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
5 5 1592 neutron-openvswi-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */
Chain neutron-openvswi-i6746373c-0 (1 references)
num pkts bytes target prot opt in out source destination
1 29 2248 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
2 2 698 RETURN udp -- * * 14.52.45.2 0.0.0.0/0 udp spt:67 udp dpt:68
3 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv4b3a1256e-07fe-4e6b-bfc5- src
4 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
5 834 34194 neutron-openvswi-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */
Chain neutron-openvswi-local (1 references)
num pkts bytes target prot opt in out source destination
Chain neutron-openvswi-o2c6951d5-e (2 references)
num pkts bytes target prot opt in out source destination
1 0 0 RETURN udp -- * * 0.0.0.0 255.255.255.255 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
2 0 0 neutron-openvswi-s2c6951d5-e all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 RETURN udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
4 0 0 DROP udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:67 udp dpt:68 /* Prevent DHCP Spoofing by VM. */
5 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
6 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
7 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
8 0 0 neutron-openvswi-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */
Chain neutron-openvswi-o6746373c-0 (2 references)
num pkts bytes target prot opt in out source destination
1 11 3488 RETURN udp -- * * 0.0.0.0 255.255.255.255 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
2 168 8824 neutron-openvswi-s6746373c-0 all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 RETURN udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
4 0 0 DROP udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:67 udp dpt:68 /* Prevent DHCP Spoofing by VM. */
5 2 168 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
6 166 8656 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
7 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
8 0 0 neutron-openvswi-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */
Chain neutron-openvswi-s2c6951d5-e (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 RETURN all -- * * 14.52.45.12 0.0.0.0/0 MAC FA:16:3E:CB:06:23 /* Allow traffic from defined IP/MAC pairs. */
2 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* Drop traffic without an IP/MAC allow rule. */
Chain neutron-openvswi-s6746373c-0 (1 references)
num pkts bytes target prot opt in out source destination
1 168 8824 RETURN all -- * * 14.52.45.10 0.0.0.0/0 MAC FA:16:3E:A8:97:6F /* Allow traffic from defined IP/MAC pairs. */
2 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* Drop traffic without an IP/MAC allow rule. */
Chain neutron-openvswi-sg-chain (4 references)
num pkts bytes target prot opt in out source destination
1 11 1904 neutron-openvswi-i2c6951d5-e all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap2c6951d5-e6 --physdev-is-bridged /* Jump to the VM specific chain. */
2 0 0 neutron-openvswi-o2c6951d5-e all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap2c6951d5-e6 --physdev-is-bridged /* Jump to the VM specific chain. */
3 865 37140 neutron-openvswi-i6746373c-0 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap6746373c-01 --physdev-is-bridged /* Jump to the VM specific chain. */
4 179 12312 neutron-openvswi-o6746373c-0 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap6746373c-01 --physdev-is-bridged /* Jump to the VM specific chain. */
5 709 67190 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0
Chain neutron-openvswi-sg-fallback (4 references)
num pkts bytes target prot opt in out source destination
1 865 44002 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* Default drop rule for unmatched traffic. */
内容过多，不适合阅读。
排查过程
我们的目标是找出不让虚拟机访问网关地址上的tftp服务的规则。 虚拟机ip是14.52.45.10，网关是14.52.45.2。
可以看到，有3个默认链：INPUT、FORWARD、OUTPUT，默认链中均有neutron开头的自定义链。背景中提到是网络包无法到达虚拟机，因此从FORWARD链中查找。
层层检索，发现源为网关的包只允许通过67``68端口udp报文(dhcp报文)
Chain neutron-openvswi-i6746373c-0 (1 references)
num pkts bytes target prot opt in out source destination
1 29 2248 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
2 2 698 RETURN udp -- * * 14.52.45.2 0.0.0.0/0 udp spt:67 udp dpt:68
3 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv4b3a1256e-07fe-4e6b-bfc5- src
4 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
5 834 34194 neutron-openvswi-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */
其他端口均作丢弃处理。
Chain neutron-openvswi-sg-fallback (4 references)
num pkts bytes target prot opt in out source destination
1 865 44002 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* Default drop rule for unmatched traffic. */
由于tftp服务器返回数据用的是随机端口，因此我们需要开通网关到虚拟机的udp协议的所有端口
#iptables -I FORWARD 1 -s 14.52.45.2 -d 14.52.45.10 -p udp -j ACCEPT
num pkts bytes target prot opt in out source destination
1 0 0 ACCEPT udp -- * * 14.52.45.2 14.52.45.10
再次从虚拟机发送tftp请求，报文就能通过了。
