转载: Optimizing the kernel for VMware

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
[*] 64-bit kernel (leave blank for x86)
 
General Setup --->
  [*] Optimize very unlikely/likely branches
 
Processor type and features --->
  Processor Family (usually Core2/Newer Xeon)
 
Bus Options --->
  < > PCCard (PCMCIA/Cardbus) support
 
Networking support --->
  [ ] Amateur Radio support
  [ ] Wireless (only if you plan on using nat EXCLUSIVELY)
 
Device Drivers --->
  Generic Driver Options
    [*] Maintain a devtmpfs filesystem to mount at /dev
 
  Misc Devices --->
    [*] VMware Balloon Driver (manages memory between VM and host)
    [*] VMware VMCI Driver (Virtual Machine Communication Interface - low-latency access to host memory bus)
 
  SCSI device support --->
    [*] SCSI low-level drivers --->
      <*> VMware PVSCSI driver support (high throughput storage adapter)
 
  [*] Fusion MPT device support --->
    <*> Fusion MPT ScsiHost drivers for SPI
 
  [*] Network device support --->
    [*] Ethernet driver support --->
      (disable every driver but this)
      [*] AMD Devices
        <*> AMD PCNet32 PCI support
    [ ] Wireless LAN (ONLY if you disabled Wireless networking support above)
    < > VMware VMXNET3 ethernet driver (PCNet32 is more than enough for most use cases - enable this only if you have spare cpu cycles to burn)
 
  Graphics support --->
    <*> Direct Rendering Manager
    < > Intel 8xx/9xx/G3x/G4x/HD Graphics
    <*> DRM driver for VMware Virtual GPU
      [*] Enable framebuffer console support under vmwgfx by default
    <*> Support for frame buffer devices
 
    Console display driver support --->
      <*> Framebuffer Console support
 
  Sound card support --->
    <*> Advanced Linux Sound Architecture --->
      [*] PCI sound devices
        <*> (Creative) Ensoniq AudioPCI 1371/1373
        < > Intel HD Audio
 
  File systems --->
    (enable only those you anticipate using)
    <*> Second extended fs support
    <*> The Extended 4 (ext4) filesystem
    <*> XFS filesystem support
    <*> Btrfs filesystem Unstable disk format
 
    Pseudo filesystems --->
      [*] Tmpfs virtual memory file system support (former shm fs)
        [*] Tmpfs POSIX Access Control Lists

refer to:
https://forums.gentoo.org/viewtopic-p-7332884.html#7332884

利用vmware调试kernel

在vmware虚拟机A上

1
2
apt install libssl-dev
apt install libncurses-dev

从文后链接中下载kernel源码,比如版本4.15.18,解压编译

1
2
3
4
make menuconfig
make
make modules_install
make install

A关机,克隆A为B,A的虚拟机设置中增加串口

1
2
3
使用命名管道:\\.\pipe\com_1
该端是服务器。
另一端是虚拟机。

B的虚拟机设置中增加串口

1
2
3
使用命名管道:\\.\pipe\com_1
该端是客户端。
另一端是虚拟机。

A以新编译的内核引导,可能事先要加大内存;B以旧内核引导。在B中运行

1
cat < /dev/ttyS1

在A中运行

1
echo Helloworld > /dev/ttyS1

如果B中回显消息,说明串口连通。
编辑A中的/boot/grub/grub.cfg,找到新编译内核启动项,在handoff后加入kgdbwait kgdboc=ttyS1,115200 nokaslr,如

1
linux /boot/vmlinuz-4.15.18 root=UUID=7ccc722d-2cbd-4597-a367-e0635333ddbf ro quiet splash $vt_handoff kgdbwait kgdboc=ttyS1,115200 nokaslr

B退出cat程序,A重启以新编译的内核引导到kdb等待状态。在B中kernel源码根目录运行

1
2
3
4
5
gdb vmlinux
set serial baud 115200
target remote /dev/ttyS1
lx-symbols
c

A中进入系统后可用下面语句触发调试

1
2
3
#echo 1 > /proc/sys/kernel/sysrq
#echo kms,kbd > /sys/module/kgdboc/parameters/kgdboc
echo g > /proc/sysrq-trigger

注1:因为新内核的KASLR机制,如果在内核启动命令行中不加入nokaslr,调试时看到不到堆栈也下不了断点。Cannot insert breakpoint。
注2:因为源地址无法访问了,有个未尝试的点摘抄下来

1
2
3
在没有编译内核的情况下,还可以直接通过修改虚拟机的启动文件.vmx,添加:
debugStub.listen.guest32 = "TRUE"
然后在调试机器中通过:target remote ip:8832(8864)来调试,ip为真实机器的IP

refer to:
https://mirrors.edge.kernel.org/pub/linux/kernel/
https://stackoverflow.com/questions/49360506/in-kgdb-i-cannot-set-the-breakpoint
https://askubuntu.com/questions/964540/gdb-qemu-cant-put-break-point-on-kernel-function-kernel-4-10-0-35
https://www.cnblogs.com/xiaofool/p/5377737.html

StrongSwan PSK RSA

四台虚拟机ABCD,三段Host Only虚拟网络vmnet2、vmnet3、vmnet4,DHCP都禁掉。

1
2
3
4
5
6
7
8
9
10
A
vmnet2: 10.1.0.10/24 gw 10.1.0.2
B
vmnet2: 10.1.0.2/24
vmnet3: 192.168.0.2/24
C
vmnet3: 192.168.0.3/24
vmnet4: 10.2.0.2/24
D
vmnet4: 10.2.0.10/24 gw 10.2.0.2

BC两台上下载编译安装strongswan 5.4.0,其版本信息可在config.h中辨认。

1
2
3
4
5
6
wget http://download.strongswan.org/strongswan-5.4.0.tar.bz2
apt-get install libgmp-dev
apt-get install libssl-dev
tar -jxvf strongswan-5.4.0.tar.bz2
./configure --sysconfdir=/etc --enable-openssl --enable-gmp --prefix=`pwd`/installed
make && make install

BC两虚拟机都指向同一个共享文件夹share。
在B中生成证书

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mkdir cert && cd cert
ipsec pki --gen --outform pem > ca.pem
ipsec pki --self --in ca.pem --dn "C=CN, O=NetworkLab, CN=NetworkLab CA" --ca --outform pem > ca.cert.pem
 
ipsec pki --gen --outform pem > sun.server.pem
ipsec pki --pub --in sun.server.pem | ipsec pki --issue --cacert ca.cert.pem --cakey ca.pem --dn "C=CN, O=NetworkLab, CN=sun.com" --san="sun.com" --flag serverAuth --flag ikeIntermediate --outform pem > sun.server.cert.pem
ipsec pki --gen --outform pem > moon.server.pem
ipsec pki --pub --in moon.server.pem | ipsec pki --issue --cacert ca.cert.pem --cakey ca.pem --dn "C=CN, O=NetworkLab, CN=moon.com" --san="moon.com" --flag serverAuth --flag ikeIntermediate --outform pem > moon.server.cert.pem
 
ipsec pki --gen --outform pem > sun.client.pem
ipsec pki --pub --in sun.client.pem | ipsec pki --issue --cacert ca.cert.pem --cakey ca.pem --dn "C=CN, O=NetworkLab, CN=client" --outform pem > sun.client.cert.pem
ipsec pki --gen --outform pem > moon.client.pem
ipsec pki --pub --in moon.client.pem | ipsec pki --issue --cacert ca.cert.pem --cakey ca.pem --dn "C=CN, O=NetworkLab, CN=client" --outform pem > moon.client.cert.pem
mv cert /mnt/hgfs/share/

B中安装证书

1
2
3
4
5
cp -r /mnt/hgfs/share/cert/ca.cert.pem /etc/ipsec.d/cacerts/
cp -r /mnt/hgfs/share/cert/moon.server.cert.pem /etc/ipsec.d/certs/
cp -r /mnt/hgfs/share/cert/moon.server.pem /etc/ipsec.d/private/
cp -r /mnt/hgfs/share/cert/sun.client.cert.pem /etc/ipsec.d/certs/
cp -r /mnt/hgfs/share/cert/sun.client.pem /etc/ipsec.d/private/

C中安装证书

1
2
3
4
5
cp -r /mnt/hgfs/share/cert/ca.cert.pem /etc/ipsec.d/cacerts/
cp -r /mnt/hgfs/share/cert/sun.server.cert.pem /etc/ipsec.d/certs/
cp -r /mnt/hgfs/share/cert/sun.server.pem /etc/ipsec.d/private/
cp -r /mnt/hgfs/share/cert/moon.client.cert.pem /etc/ipsec.d/certs/
cp -r /mnt/hgfs/share/cert/moon.client.pem /etc/ipsec.d/private/

B中配置
/etc/ipsec.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
config setup
	# strictcrlpolicy=yes
	# uniqueids = no
conn %default
	ikelifetime=60m
	keylife=20m
	rekeymargin=3m
	keyingtries=1
	keyexchange=ikev2
	mobike=no
conn net-net-rsa
	left=192.168.0.2
	leftcert=moon.server.cert.pem
	leftsubnet=10.1.0.0/24
	leftid=@moon.com
	leftfirewall=yes
	right=192.168.0.3
	rightsubnet=10.2.0.0/24
	rightid=@sun.com
	auto=add
conn net-net-psk
	keyexchange=ikev1
	authby=secret
	left=192.168.0.2
	leftsubnet=10.1.0.0/16
	leftid=@xxx.server.com
	leftfirewall=yes
	right=192.168.0.3
	rightsubnet=10.2.0.0/16
	rightid=@yyy.server.com
	ike=aes-sha1-modp1024
	esp=aes-sha1-modp1024
	auto=route
	type=tunnel

/etc/ipsec.secrets

1
2
@xxx.server.com @yyy.server.com : PSK hello
: RSA moon.server.pem

/etc/strongswan.conf

1
2
3
4
5
6
7
charon {
	load_modular = yes
	plugins {
		include strongswan.d/charon/*.conf
	}
}
include strongswan.d/*.conf

C中配置
/etc/ipsec.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
config setup
	# strictcrlpolicy=yes
	# uniqueids = no
conn %default
	ikelifetime=60m
	keylife=20m
	rekeymargin=3m
	keyingtries=1
	keyexchange=ikev2
	mobike=no
conn net-net-rsa
	left=192.168.0.3
	leftcert=sun.server.cert.pem
	leftsubnet=10.2.0.0/24
	leftid=@sun.com
	leftfirewall=yes
	right=192.168.0.2
	rightsubnet=10.1.0.0/24
	rightid=@moon.com
	auto=add
conn net-net-psk
	keyexchange=ikev1
	authby=secret
	left=192.168.0.3
	leftsubnet=10.2.0.0/16
	leftid=@yyy.server.com
	leftfirewall=yes
	right=192.168.0.2
	rightsubnet=10.1.0.0/16
	rightid=@xxx.server.com
	ike=aes-sha1-modp1024
	esp=aes-sha1-modp1024
	auto=route
	type=tunnel

/etc/ipsec.secrets

1
2
@xxx.server.com @yyy.server.com : PSK hello
: RSA sun.server.pem

/etc/strongswan.conf

1
2
3
4
5
6
7
charon {
	load_modular = yes
	plugins {
		include strongswan.d/charon/*.conf
	}
}
include strongswan.d/*.conf

到此,BC两台机器中都运行

1
2
3
4
5
6
echo 1 > /proc/sys/net/ipv4/ip_forward
ipsec restart --nofork
#如果要以rsa方式
ipsec up net-net-rsa
#如果要以psk方式
Ipsec up net-net-psk

最后在A中ping机器D。

refer to:
https://blog.csdn.net/puppylpg/article/details/64918562
http://www.hqyman.cn/post/543.html
https://www.cnblogs.com/hugetong/p/10150992.html

安装程序无法自动安装 Virtual Machine Communication Interface Sockets (VSock)

http://www.catalog.update.microsoft.com/search.aspx?q=kb4474419
下载kb4474419 win7 x64版珍藏!

refer to:
https://tieba.baidu.com/p/6031541992
https://kb.vmware.com/s/article/78708
https://support.microsoft.com/en-us/topic/sha-2-code-signing-support-update-for-windows-server-2008-r2-windows-7-and-windows-server-2008-september-23-2019-84a8aad5-d8d9-2d5c-6d78-34f9aa5f8339

compile ofp over dpdk in vmware

1. get software

1
2
git libtool automake build-essential pkg-config libssl-dev doxygen libconfig-dev libnuma-dev libpcap-dev net-tools
ln -s /usr/bin/python3 /usr/bin/python

2. compile dpdk

1
git clone http://dpdk.org/git/dpdk-stable --branch 19.11 --depth 1 dpdk-19.11

run dpdk-19.11/usertools/dpdk-setup.sh
>>> select and run "x86_64-native-linuxapp-gcc"
>>> Exit Script

3. compile odp-dpdk

1
git clone https://github.com/OpenDataPlane/odp-dpdk.git

odp-dpdk/m4/odp_dpdk.m4

1
2
3
4
5
6
7
8
9
10
11
12
13
# _ODP_DPDK_LEGACY(PATH, ACTION-IF-FOUND, ACTION-IF-NOT-FOUND)
# ------------------------------------------------------------------------
# Locate DPDK installation
AC_DEFUN([_ODP_DPDK_LEGACY], [dnl
    DPDK_CFLAGS="-isystem $1/include"
    DPDK_LIB_PATH="$1/lib"
    DPDK_LDFLAGS="-L$DPDK_LIB_PATH"
    AS_IF([test -r "$DPDK_LIB_PATH"/libdpdk.so], [dnl
        DPDK_RPATH="-Wl,-rpath,$DPDK_LIB_PATH"
        DPDK_RPATH_LT="-R$DPDK_LIB_PATH"
        DPDK_SHARED=yes],
        [test ! -r "$DPDK_LIB_PATH"/libdpdk.a], [dnl
        AC_MSG_FAILURE([Could not find DPDK])])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#!/bin/sh
 
DPDK_PATH=/home/eu/work/dpdk/dpdk-19.11/x86_64-native-linuxapp-gcc
 
cd odp-dpdk
./bootstrap
#--enable-debug --enable-debug-print
#--without-openssl
./configure --with-dpdk-path=${DPDK_PATH} \
                --prefix=`pwd`/installed \
                --enable-debug=full \
                --enable-helper-debug \
                --enable-debug-print \
                --enable-helper-debug-print \
                --disable-shared
make install

4. compile ofp

1
git clone https://github.com/OpenFastPath/ofp.git
1
2
3
4
5
6
7
8
9
10
11
12
13
#!/bin/sh
 
ODP_DPDK_PATH=`pwd`/odp-dpdk/installed
 
cd ofp
./bootstrap
./configure --with-odp=${ODP_DPDK_PATH} \
        --with-odp-lib=odp-dpdk \
        --disable-shared \
        --enable-debug=yes \
        --enable-sp=no \
        --prefix=`pwd`/installed
make install

5. setup dpdk

1
2
3
4
cd dpdk-19.11/x86_64-native-linuxapp-gcc/kmod
modprobe uio
insmod igb_uio.ko
ifconfig ens33 down

run dpdk-19.11/usertools/dpdk-setup.sh
>>> select and run "Setup hugepage mappings for non-NUMA systems", enter 128
>>> select and run "Bind Ethernet/Baseband/Crypto device to IGB UIO module", will display like
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens33 drv=e1000 unused=igb_uio,vfio-pci
>>>>>> enter 02:01.0
>>> Exit Script

6. test

let the vmware guest machine use NAT.

use ipconfig check ip settings in host machine. for example, my ip for "VMware Network Adapter VMnet8" is 192.168.204.1.

ofp/example/webserver/ofp.cli

1
2
3
debug 0
loglevel set debug
ifconfig fp0 192.168.204.33/24

ofp/example/webserver/www/aa.txt

1
Hello, the world!

ofp/example/webserver/run.sh

1
2
export www_dir=`pwd`/www
./webserver -i 0 -f ofp.cli

using web browser access

1
http://192.168.204.33:2048/aa.txt

vmware linux共享文件夹未启动

1
2
3
4
# 若vmhgfs-fuse -e执行成功时执行
vmhgfs-fuse -o subtype=vmhgfs-fuse,allow_other /mnt/hgfs/
# mvhgfs-fuse -e 执行失败时执行
mount -f vmhgfs .host:/ /mnt/hgfs

如果还不行,比如更新内核后,就要重新加载linux.iso重装VMWareTools。

refer to: https://www.cnblogs.com/woodyoilove/p/8479458.html

Failed to format -2147024809

直接用adk里的MakeWinPEMedia写winpe到U盘里报标题这个错误,所以我们换一种思路,用MakeWinPEMedia生成iso,再在linux环境将iso擦写到U盘里。

先以管理员方式运行“部署和映像工具环境”
输入

1
MakeWinPEMedia /ISO d:\euhat\WinPE_amd64 d:\winpe.iso

成功后,将winpe.iso拷贝到ubuntu linux虚拟机或实机里面。
我们现在以vmware的ubuntu虚拟机为例,为了使虚拟机正常识别U盘,需要做两件事

  1. 以管理员方式运行vmware player或workstation。
  2. ubuntu虚拟机设置里USB控制器兼容性中选择正确的版本,新买的机器一般选USB 3.0以上。这一步骤若选错,在vmware工具条上将U盘连接到虚拟机里时会提示“无法连接到理想的主机控制器。将尝试将该设备连接到可用的最佳主机控制器”不成功的错误,那时再重复此步骤选择其它USB控制器版本试试。

启动ubuntu虚拟机,打开终端以root身份运行

1
fdisk -l

通过看容量找到U盘的设备路径,如我这里是/dev/sdb,则再运行

1
2
dd if=/home/euhat/Desktop/winpe.iso of=/dev/sdb
sync

这样winpe就写入U盘了,正常从虚拟机、Windows卸载U盘后,再插到机器上用U盘引导试试!

VMWare磁盘读写提速

VMWare不生成vmem文件的方法是在.vmx文件中增加两行:

1
2
sched.mem.pshare.enable = "FALSE"
mainMem.useNamedFile = "FALSE"

refer to:
https://blog.csdn.net/shanzhizi/article/details/8293638
http://www.360doc.com/content/15/1120/11/73007_514517328.shtml
http://www.guyiren.com/archives/3210
https://blog.csdn.net/mnmnwq/article/details/79557305
https://blog.51cto.com/kitzk/553493
https://segmentfault.com/q/1010000002954977
https://bbs.kafan.cn/thread-1355085-1-1.html

本机代理到虚拟机

由于局域网限MAC地址,VmWare虚拟机不能直接连接物理网络,只能NAT模式,那么怎么让别人的电脑连接到本机虚拟机里的服务程序呢?

比如虚拟机里的服务程序IP为192.168.38.132,端口为5454,

本机IP为192.168.1.67,现在我们做一个代理,让别人能访问到192.168.38.132:5454。

在本机,以管理员模式运行cmd,输入

1
2
3
4
5
netsh
interface
portproxy
 
add v4tov4 listenaddress=192.168.1.67 listenport=5454 connectaddress=192.168.38.132 connectport=5454

记住,再禁掉本机防火墙!
此时,别人的电脑就可以访问此虚拟机里的服务程序了。

当不再需要此代理时,在本机,以管理员模式运行cmd,输入

1
2
3
4
5
netsh
interface
portproxy
 
delete v4tov4 listenaddress=192.168.1.67 listenport=5454

注意:微软的netsh只能转发tcp包,没有实现转发udp包。

参考:https://blog.csdn.net/mergerly/article/details/50747930