lost and found ( for me ? )

KVM : hot-add PCI devices ( NIC )


[ KVM host info ]
# tail -1 /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 11.04"

# uname -r
2.6.38-11-server

# kvm --version
QEMU emulator version 0.14.0 (qemu-kvm-0.14.0), Copyright (c) 2003-2008 Fabrice Bellard

[ Requirements ]

in my testing environment :

KVM host needs kvm-pxe. ( you can install it via apt-get )
VM : Scientfic Linux 6.1 needs load
VM : Ubuntu 10.04 needs to load acpiphp module.

Here are trial and error logs I did.

[ Preparation on KVM host ]

I need to install kvm-pxe on KVM host.
When I haven’t installed kvm-pxe on KVM host , I see the following error when adding a NIC and I can’t add a NIC to a VM.
virsh # qemu-monitor-command --hmp ubuntu-vm1 'pci_add auto nic model=virtio,macaddr=11:22:33:44:55:66'
pci_add_option_rom: failed to find romfile "pxe-virtio.bin"
OK domain 0, bus 0, slot 5, function 0

install kvm-pxe on KVM host.
# apt-get install kvm-pxe

[ add/delete PCI devices ( NIC ) to a guest VM running Scientific linux 6.1 ]

Seen from http://www.linux-kvm.org/page/Hotadd_pci_devices , this describes that a VM needs to load acpiphp and pci_hotplug modules , however I could add PCI devices without loading the above modules on a VM running SL 6.1.

Let’s add a NIC to SL6.1.
Here’s an output of “lsmod” on SL6.1.
# lsmod
Module                  Size  Used by
autofs4                27683  3
sunrpc                242213  1
ipt_REJECT              2349  2
nf_conntrack_ipv4       9440  2
nf_defrag_ipv4          1449  1 nf_conntrack_ipv4
iptable_filter          2759  1
ip_tables              17765  1 iptable_filter
ip6t_REJECT             4562  2
nf_conntrack_ipv6       8650  2
nf_defrag_ipv6         12148  1 nf_conntrack_ipv6
xt_state                1458  4
nf_conntrack           79611  3 nf_conntrack_ipv4,nf_conntrack_ipv6,xt_state
ip6table_filter         2855  1
ip6_tables             19424  1 ip6table_filter
ipv6                  322291  27 ip6t_REJECT,nf_conntrack_ipv6,nf_defrag_ipv6
dm_mirror              14067  0
dm_region_hash         12136  1 dm_mirror
dm_log                 10120  2 dm_mirror,dm_region_hash
uinput                  8182  0
sg                     30090  0
microcode             112781  0
virtio_balloon          4281  0
virtio_net             15741  0
i2c_piix4              12574  0
i2c_core               31274  1 i2c_piix4
ext4                  359671  1
mbcache                 7918  1 ext4
jbd2                   88768  1 ext4
sr_mod                 16194  0
cdrom                  39769  1 sr_mod
virtio_blk              5692  3
ata_generic             3611  0
pata_acpi               3667  0
ata_piix               22652  0
virtio_pci              6653  0
virtio_ring             7169  4 virtio_balloon,virtio_net,virtio_blk,virtio_pci
virtio                  4824  4 virtio_balloon,virtio_net,virtio_blk,virtio_pci
dm_mod                 75539  2 dm_mirror,dm_log

before adding a NIC.
SL6.1 has one NIC. ( qemu monitor )
virsh # qemu-monitor-command --hmp SL6-vm1 'info network'
Devices not on any VLAN:
 hostnet0: fd=18 peer=net0
 net0: model=virtio-net-pci,macaddr=52:54:00:11:18:f1 peer=hostnet0

add a NIC ( qemu monitor )
virsh # qemu-monitor-command --hmp SL6-vm1 'pci_add auto nic model=virtio'
OK domain 0, bus 0, slot 4, function 0

qemu monitor
virsh # qemu-monitor-command --hmp SL6-vm1 'info network'
VLAN 0 devices:
 virtio-net-pci.0: model=virtio-net-pci,macaddr=52:54:00:12:34:56
Devices not on any VLAN:
 hostnet0: fd=18 peer=net0
 net0: model=virtio-net-pci,macaddr=52:54:00:11:18:f1 peer=hostnet0

on VM
# ifconfig -a | grep eth | awk '{print $1}'
eth0
eth1  <- added

dmesg on VM
virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)
virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11
 alloc irq_desc for 29 on node -1
 alloc kstat_irqs on node -1
virtio-pci 0000:00:04.0: irq 29 for MSI/MSI-X
 alloc irq_desc for 30 on node -1
 alloc kstat_irqs on node -1
virtio-pci 0000:00:04.0: irq 30 for MSI/MSI-X
 alloc irq_desc for 31 on node -1
 alloc kstat_irqs on node -1
virtio-pci 0000:00:04.0: irq 31 for MSI/MSI-X

delete a NIC eth1 ( qemu monitor )
virsh # qemu-monitor-command --hmp SL6-vm1 'pci_del pci_addr=4

virsh # qemu-monitor-command --hmp SL6-vm1 'info network'
VLAN 0 devices:
Devices not on any VLAN:
 hostnet0: fd=18 peer=net0
 net0: model=virtio-net-pci,macaddr=52:54:00:11:18:f1 peer=hostnet0

on VM
# ifconfig -a | grep eth | awk '{print $1}'
eth0

[ add/delete PCI devices ( NIC ) to a guest VM running Ubuntu 10.04 ]

I built a Ubuntu 10.04 VM with vmbuilder.

An output of “lsmod” on VM.
# lsmod
Module                  Size  Used by
fbcon                  35102  71
tileblit                1999  1 fbcon
font                    7557  1 fbcon
bitblit                 4707  1 fbcon
serio_raw               3978  0
softcursor              1189  1 bitblit
psmouse                63245  0
vga16fb                11385  1
virtio_balloon          2835  0
vgastate                8961  1 vga16fb
floppy                 53080  0

VM has one NIC. ( qemu-monitor )
virsh # qemu-monitor-command --hmp ubuntu-vm1 'info network'
Devices not on any VLAN:
 hostnet0: fd=18 peer=net0
 net0: model=virtio-net-pci,macaddr=52:54:00:0a:7e:0a peer=hostnet0

add a NIC ( qemu-monitor )
virsh # qemu-monitor-command --hmp ubuntu-vm1 'pci_add auto nic model=virtio,mac
addr=11:22:33:44:55:66'
OK domain 0, bus 0, slot 5, function 0

virsh # qemu-monitor-command --hmp ubuntu-vm1 'info network'
VLAN 0 devices:
 virtio-net-pci.0: model=virtio-net-pci,macaddr=11:22:33:44:55:66
Devices not on any VLAN:
 hostnet0: fd=18 peer=net0
 net0: model=virtio-net-pci,macaddr=52:54:00:0a:7e:0a peer=hostnet0

nnn , there isn’t an additional NIC on Ubuntu VM.
# ifconfig –a | grep eth | awk '{print $1}'
eth0

load acpiphp and pci_hotplug modules and try it again.
It seems that there’s no pci_hotplug module in Ubuntu.
so load only acpipho module.
# modprobe acpiphp
# modprobe pci_hotplug
FATAL: Module pci_hotplug not found.

# lsmod
Module                  Size  Used by
acpiphp                18316  0
fbcon                  35102  71
tileblit                1999  1 fbcon
font                    7557  1 fbcon
bitblit                 4707  1 fbcon
softcursor              1189  1 bitblit
serio_raw               3978  0
vga16fb                11385  1
psmouse                63245  0
vgastate                8961  1 vga16fb
virtio_balloon          2835  0
floppy                 53080  0

try again.
virsh # qemu-monitor-command --hmp ubuntu-vm1 'pci_add auto nic model=virtio,macaddr=11:22:33:44:55:77'
OK domain 0, bus 0, slot 6, function 0

NICs has been added.
# ifconfig -a | grep eth | awk '{print $1}'
eth0
eth1
eth3

delete NICs ( eth1 , eth3 )
virsh # qemu-monitor-command --hmp ubuntu-vm1 'pci_del pci_addr=5'
virsh # qemu-monitor-command --hmp ubuntu-vm1 'pci_del pci_addr=6'

NICs have been removed.
# ifconfig -a | grep eth | awk '{print $1}'
eth0

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.