WORK IN PROGRESS - Your Inputs & Thoughts
To Technology Laboratory - To Technology Laboratory - Overview Projects - To New Infrastructure Host - Bern - To The Infrastructure Host Project
Basics Infra-ch01
Design & Development of Set-Up
by Daniel Black
Purpose
- Provide hosting for virtual machines
- provide an apt-proxy services for repositories
Physical Location
- This system is located on a Debian Lenny machine and serves that the virtual host machine.
Logical location
- IP: 212.103.75.4
- Internal IP: 192.168.75.4/28
Applicable Documentation
Administration
Primary: Daniel Black
Secondary: Philipp Gühring
OS setup
- Debian Lenny
- RAM: 8G
- Swap: 4G
- Root: 20G /dev/sda1
- Rest: 121G /dev/sda3 LVM Partition
- sda3 mapped to /dev/drbd1 (/etc/drbd.conf - replicated to cacerttest-ch01)
- drbd1 used in vg0 (lvm volume group)
- /dev/vg0/* used by virtual machines
Services
Listening services
port
service
access origin
purpose
TCP/17888
SSH
all
SSH access for remote administration
TCP/14442
backup agent
backupserver
backup
TCP/9999
apt-proxy
localnet
proxy server for apt repositories (no routing between local and public networks)
UDP/53
DNS
localnet
local DNS proxy
TCP/53
DNS
localnet
local DNS proxy
UDP/161
SNMP
localnet
snmp information for cacti
DNS
Running Services
Service
Started from
/opt/rbu/agent
/etc/init.d/bru-backup-agent
dbus
init.d
cron
init.d
syslog
init.d
snmpd
init.d
ssh
init.d
puppet
init.d
kernelopps
init.d
[drbd1_worker]
init.d
apt-proxy
init.d
kvm/ libvirtd
init.d/libvirt-bin
dnsmasq
init.d
uptrack
init.d/cron.d
Attached Systems
System
IP
192.168.75.5
212.105.75.5
212.105.75.12
Outbound network connections
- DNS (53)
HTTP (80) package updates http://mirror.switch.ch http://security.debian.org http://backports.org http://www.ksplice.com (/etc/apt-proxy/apt-proxy-v2.conf)
Security
- Privileged Access: Daniel/Philipp
Risk assessments on critical packages
- KVM - stable enough for redhat
- DRBD - mature produce - widespread use - still running older debian lenny stable version though
- LVM - very common use
- KSplice - large list of unmodified upstream patches - only used for kernel vulnerabilities - only other option is a more managed reboot which isn't desireable on a visualization host machine.
Common Tasks
KVM Creating a VM
hostname=<hostname> lvcreate -L 2G -n $hostname vg0 virt-install --connect qemu:///system -n $hostname -r 512 --vcpus=1 --disk path=/dev/vg0/$hostname -c ~/debian-504-amd64-netinst.iso --vnc --noautoconsole --os-type linux --os-variant debianLenny --accelerate --hvm --network=bridge:<br0 or br1> # br0 - public IP / br1 - private IP # look for new listening socket like 590X (local) ssh -L 590X:127.0.0.1:590X infra-ch01 -p 17888 (local) vncviewer :X SecurityTypes=None # should have Debian GUI install now # -> Install # -> English # -> Other -> Europe -> Switzerland # (Keymap) -> American English # Network - manual address. netmask is 255.255.255.240, gw is 212.103.75.1 for public, nameserver 212.103.64.17 (will eventually bet set to 192.168.75.4 once local routing done) # hostname - <hostname> # domain name - cacert.org # (DISK LAYOUT) - Guided - Use entire disk -> vda -> All files in one partition (TODO review later) -> Finish partition and write (should show a root dir and a swap partition) # root password - something rememberable # user account - your name # debian archive - 192.168.75.4:9999 for non-public ips. Switzerland -> mirror.switch.ch otherwise -> no proxy # standard install # write grub to mbr # reboot... (vnc will disconnect) (on infra-ch01 again) virsh edit <hostname> remove cdrom xml defination ensure boot dev=hd virsh start <hostname> (local) restart vncviewer # public IP (stored in /etc/network/if-up.d/localroute) ip route add to 192.168.75.0/28 via 212.103.75.4 echo ip route add to 192.168.75.0/28 via 212.103.75.4 >> /etc/network/if-up.d/localroute # or private IP: ip route add to 212.103.75.0/28 via 192.168.75.4 echo ip route add to 212.103.75.0/28 via 192.168.75.4 >> /etc/network/if-up.d/localroute chmod a+x /etc/network/if-up.d/localroute echo nameserver 192.168.75.4 >> /etc/resolv.conf apt-get update apt-get upgrade apt-get install sudo puppet etckeeper git openssh-server apt-get remove --purge portmap exim4 nfs-common exim4-config exim4-base exim4-daemon-light laptop-detect etckeeper init (local) scp ~/.ssh/id_dsa.pub (publicip):~/authorized_keys mkdir ~/.ssh mv authorized_keys .ssh vi /etc/ssh/sshd_config - port 17888, passwordAuthentication no, PermitRootLogin no sudo /etc/init.d/ssh restart echo 192.168.75.5 puppet >> /etc/hosts sudo /etc/init.d/puppet start etckeeper commit (puppetmaster) puppetca --list (puppetmaster) puppetca --sign <hostname.cacert.org>
forgotten root password
virsh shutdown <hostname>; fdisk -l; mount /dev/vg0/<hostname> -o offset=32256 /mnt; mount --bind {/dev,/dev/pts,proc} /mnt/ {/dev,/dev/pts,proc}; chroot /mnt/; passwd
Controlling VMs
- virsh list
- virsh start {domain}
- virsh shutdown {domain}
IPtables
- Change live
/etc/network/if-post-down.d/iptables && etckeeper commit to save
Critical Configuration items
DRBD handling
- status cat /proc/drbd
- command and control - drbdadm
/etc/drbd.conf
- Configuration of replication.
/etc/network/iptables.save
- local firewall rules
- saved by /etc/network/if-post-down.d/iptables
- started by /etc/network/if-post-down.d/iptables
/etc/network/interfaces
- bridge interface definations
- br0 - public bridge shared with kvm
- br1 - private bridge shared with kvm
Changes
Planned
Security upgrade
- selinux on host?
- more restricted firewall rule - currently depending on stuff not listening public or like bru - blocked by iptables as required.
Inputs & Thoughts
YYYYMMDD-YourName
Text / Your Statements, thoughts and e-mail snippets, Please
YYYYMMDD-YourName
Text / Your Statements, thoughts and e-mail snippets, Please
Category or Categories