~~Zhuang Shang-Yi~~: 2007/09

2007/09/26

Linking vs. Loading

Program loading. loading 會將程式從硬碟上複製到到記憶體上, 好讓程式可以執行. 有時候loading只要做複製的動作, 但有時候需要做配置記憶體的動作或設定protection bits.

Relocation. Compilers和assemblers通常將每個編譯好的程式(object code)的記憶體位置設定從零開始. 但是只有很少的電腦可以讓程式從
零開始. 當有多個程式包含有許多副程式的時候, 所有副程式需要被配置在non-overlapping的位置. Relocation就是負責幫程式指定(assigning)
load addresses. 在很多系統中, relocation通常不只發生一次.

Symbol resolution. 當程式要從其他副程式建置的時候. 程式references到副程式通常是用symbols. 例如一個程式要使用sqrt的副程式.會直接寫sqrt這個symbols. 而linker會將紀錄location assigned to sqrt in the library. 然後patching caller's(主程式) object code.

雖然linking和loading有許多重複的地方. 但是通常還是將他們區分為. 做 program loading 的是loader, 而做symbol resolution的是linker. 兩者都能做relocation.

Two-Pass Linking
linker負責處理被compiler或assembler編譯過的object files. 對linker來說就是一個input file. 而每個input file 包含了a set of segments.
每個input file 也至少包含一個symbol table. 有些symbol table是exported的, 用來給其他檔案使用. 當一個linker執行的時候, 會先掃過
一次input files確定每個segments的大小, 和蒐集每個symbol的 definitions 和 references. 然後產生一個segment table列出所有在 input
file被定義的segments. 還有一個symbol table包含export 和import的symbols. 接下來linker會指定記憶體位置給symbols, 決定segments
位置的大小. 第二次pass, 使用第一次pass的資訊產生relocation code放到output file. 而output file 通常會有header information, relocated
segments, and symbol table information. 如果是dynamic linking,symbol table會包含run-time linker所需要的資訊.

Relocation and code modification是主要linker&loader的工作.

#!/bin/bash
#---------------mini-rc.lvs_dr-director------------------------
#set ip_forward OFF for vs-dr director (1 on, 0 off)
#cat       /proc/sys/net/ipv4/ip_forward
echo "0" >/proc/sys/net/ipv4/ip_forward

#director is not gw for realservers: leave icmp redirects on
#echo 'setting icmp redirects (1 on, 0 off) '
echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects
#cat       /proc/sys/net/ipv4/conf/all/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects
#cat       /proc/sys/net/ipv4/conf/default/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/eth0/send_redirects
#cat       /proc/sys/net/ipv4/conf/eth0/send_redirects

#add ethernet device and routing for VIP 192.168.1.110
ifconfig eth0:110 192.168.80.110 broadcast 192.168.80.110 netmask 255.255.255.255
/sbin/route add -host 192.168.80.110 dev eth0:110
#listing ifconfig info for VIP 192.168.1.110
/sbin/ifconfig eth0:110

#check VIP 192.168.1.110 is reachable from self (director)
#/bin/ping -c 1 192.168.1.110
#listing routing info for VIP 192.168.1.110
/bin/netstat -rn

#setup_ipvsadm_table
#clear ipvsadm table
/sbin/ipvsadm -C
#installing LVS services with ipvsadm
#add telnet to VIP with round robin scheduling
/sbin/ipvsadm -A -t 192.168.80.110:http -s wrr

#forward telnet to realserver using direct routing with weight 1
/sbin/ipvsadm -a -t 192.168.80.110:http -r 192.168.80.62 -g -w 1
/sbin/ipvsadm -a -t 192.168.80.110:http -r 192.168.80.61 -g -w 1
#check realserver reachable from director
#ping -c 1 192.168.1.12

#forward telnet to realserver using direct routing with weight 1
#/sbin/ipvsadm -a -t 192.168.1.110:http -r 192.168.1.12 -g -w 1
#check realserver reachable from director
#ping -c 1 192.168.1.12

#displaying ipvsadm settings
/sbin/ipvsadm

#not installing a default gw for LVS_TYPE vs-dr
#---------------mini-rc.lvs_dr-director------------------------

後端設定檔:

#!/bin/bash
#----------mini-rc.lvs_dr-realserver------------------
#installing default gw 192.168.1.254 for vs-dr
#/sbin/route add default gw 192.168.1.254
#showing routing table
#/bin/netstat -rn
#checking if DEFAULT_GW 192.168.1.254 is reachable
#ping -c 1 192.168.1.254

#set_realserver_ip_forwarding to OFF (1 on, 0 off).
echo "0" >/proc/sys/net/ipv4/ip_forward
#cat       /proc/sys/net/ipv4/ip_forward

#looking for DIP 192.168.1.9
#ping -c 1 192.168.1.9

#looking for VIP (will be on director)
#ping -c 1 192.168.1.110

#install_realserver_vip
/sbin/ifconfig lo:110 192.168.80.110 broadcast 192.168.80.110 netmask 0xffffffff up
#ifconfig output
/sbin/ifconfig lo:110
#installing route for VIP 192.168.80.110 on device lo:110
/sbin/route add -host 192.168.80.110 dev lo:110
#listing routing info for VIP 192.168.80.110
/bin/netstat -rn

#hiding interface lo:110, will not arp
echo "1" >/proc/sys/net/ipv4/conf/all/hidden
#cat       /proc/sys/net/ipv4/conf/all/hidden
echo "1" >/proc/sys/net/ipv4/conf/lo/hidden
#cat       /proc/sys/net/ipv4/conf/lo/hidden

#----------mini-rc.lvs_dr-realserver------------------

2007/09/25

L&L memo

Linker & loader 的基本工作是將抽象的名字和更具體的名字綁在一起...
換句話說, 這可以讓程式設計師用比較抽象的表示方法寫程式.

歷史的觀點

早期的程式都要用machine language寫. 程式設計師先用比較抽象的語法寫在紙上之後,
再用"手動"的方式轉成machine language. 再放到電腦裡面跑. 如果一個變數要加減的話,
程式設計師需要自己指定位置.
後來,指定位置的動作交給了assemblers, 於是不用在指定位置,但是如果程式改變的話,
當然還是要再重新編譯.

1947年, John Mauchly 寫了loading programs, relocation and library search.
這程式比assembler還要早出現. The relocating loader允許每個副程式從位置
零的地方開始執行, 而把實際上要執行的位置延遲到此副程式被呼叫的時候才決定.

在operating system出現之前, 程式可以被寫在固定的位置, 因為每個位置都是available的.
但是有OS之後, 程式必須和OS or other programs分享記憶體空間. 這代表著程式沒有辦法
知道哪些記憶體位置是空的, 直到程式被載入記憶體的時候才能確定. 因此linker和loader有
各自的工作了. Linker負責一部份的address binding,指定相對位置(relative addresses)給程式.
而Loader則做最後的relocation步驟-- 指定實際位置(actual addresses).

後來, 當程式越來越大, 甚至超過可以使用的記憶體空間的時候, linkers提供了overlays的功能.
可以讓程式設計師將一個程式中不同部分使用同樣的記憶體位置. 這種方法還是被使用在一些
memory-limited embedded環境中.

既然有了hardware relocation 和 virtual memory的出現, linkers 和 loaders的工作就變的比較簡單. 因為每個程式又可以有一整個記憶體空間. 程式可以被載入到固定的記憶體位置. Hardware
relocation將所有load-time relocation的工作包辦. 然而, 有hardware relocation的電腦上通常都
會在同一時間執行多個程式, 而且常會執行相同的程式. 當電腦跑多個相同的程式的時候, 有些
部分的程式碼是不會變動的, 而有些是會在執行的時候改變. 將這兩種變數區分出來的話(one
section for read-only code and another section for writable data), 可以節省許多記憶體空間.

另外, full-fledged dynamic linking, 將指定位置的工作延遲到第一次被呼叫的時候. 可以在程式
執行的期間才bind to library, loading libraries in the middle of program execution. 就像是MS的
Dynamically linked libraries (DLLs).

2007/09/23

Learning memo

Relocation

In computer science, relocation is the process of replacing symbolic references or names of libraries with actual usable addresses in memory before running a program.It is typically done by the linker during compilation, although it can be done at run-time by a loader.

From wikipeida.

Prelink

Most common applications make use of shared libraries. These shared libraries need to be loaded into memory at runtime and the various symbol references need to be resolved. For most small programs this dynamic linking is very quick. But for programs written in C++ and that have many library dependencies, the dynamic linking can take a fair amount of time.

On most systems, libraries are not changed very often and when a program is run, the operations taken to link the program are the same every time. Prelink takes advantage of this by carrying out the linking and storing it in the executable, in effect prelinking it.

Prelinking can cut the startup times of applications. For example, a typical KDE program's loading time can be cut by as much as 50%. The only maintenance required is re-running prelink every time a library is upgraded for a pre-linked executable.

From gentoo

2007/09/21

SIC Simplified Instructional Computer

今天早上的系統程式講了第一章
老師秀了一下SIC系統

我也在自己電腦試了一下XD
在原本的rohan.sdsu.edu上面抓的sic v1.6沒辦法compile過

所以又在google上找了一下, 發現一個1.5版本的sic
就可以run了.. Orz

1. SIC
http://studentweb.ncnu.edu.tw/95213532/sic.tgz
跑了一下test程式可是要copy的東西很繁瑣直接寫到shell script裡

#!/bin/sh
/bin/cp sictest.asm SRCFILE
/usr/bin/fpc sicasm.pas
/bin/cp sictest.obj DEVF2
/bin/cp sicboot.obj DEV00
/bin/cp sicldr.obj DEVF1
/bin/cp sictest.dat DEVF3
./sicsim

2. 習題
另外還寫了一下習題(2)
write a sequence of instructions for sic/xe to set ALPHA equal to
4*BETA - 9. Assume that ALPHA and BETA are defined as in Fig.
1.3(b). Use immediate addressing for the constants.

LDS #4
LDA BETA
MUL S
SUB #9
STA ALPHA

不知道對不對下禮拜再看看老師有沒有講解

2007/09/19

Gentoo: modification time in the future

安裝完Gentoo之後常會因為設定時間的關係出現

"One of the files in /etc/{conf.d,init.d} or /etc/rc.conf has a modification time in the future"

的錯誤訊息.

So here is some help, if you have the same little annoyance.

Here it was the case, that the file in /etc/conf.d and /etc/init.d had no timestamp (you can check with ls -lR).

You can solve this by a hearty touch /etc/{conf.d,init.d}/*; touch /etc/rc.conf.

Solution found at forums.gentoo.org

Shell script "read file" howto

To read file in a shell script is very simple.

Referenced from http://ka1fsb.home.att.net/shfio.html

Reading Data into a File All at Once.

#----- The file to read
        myFile="/root/somefile"

#----- The "big" data variable
        myData=""

#----- Now the read
        myData=`cat $myFile`

#----- Show that the data is really in the variable...
#----- This is in the same format as the orginal file, new lines preserved
        echo "$myData"

#----- Show the data in non-quoted format, the space becomes the separator
        echo $myData

If you want to read data one line at a time, try this..

#----- The file to read
        myFile="/root/somefile"

#----- The line data variable
        myLine=""

#----- Loop to read file data content
        while [ 1 ]
        do
                read myLine || break
                echo "$myLine"

        done < $myFile

2007/09/14

U-Boot & Crosstool Install

CrossTool 可以很容易地建立起 gcc/glibc cross-toolchains.

更詳細的內容請參考 http://kegel.com/crosstool/

1. 安裝cross toolchain

下載最新版本的crosstool, 目前最新版本是crosstool-0.43.tar.gz

解壓縮

# tar zxvf crosstool-0.43.tar.gz

建立資料夾放置編譯完成的程式

sudo mkdir /opt/crosstool
sudo chown $USER /opt/crosstool

進入目錄裡選擇要安裝的目標版種類. 這裡以arm為例.

而且要在非root的使用者下安裝.

另外, 要編譯u-boot 1.2.0, 要使用softfloat得cross tool才可以.

# cd crosstool-0.43

# ./demo-arm-softfloat.sh

接下來就會開始先設定一些環境變數, 下載需要的檔案, 進行編譯動作.

在下載linux核心的時候會特別慢, 因為是連到國外. 建議改成國內的鏡像站.

getandpatch.sh是用來下載檔案的shell script

直接修改裡面的網址就可以了.

安裝完之得toolchain會放在

/opt/crosstool/gcc-3.4.5-glibc-2.3.6/arm-softfloat-linux-gnu/bin

可以將上面路徑加入$PATH還境變數.

或者你想要登入都有這個工具可以在

/etc/profile裡面加入設定

2. U-Boot

安裝完toolchain之後就可以開始製作u-boot了

這裡使用arm920t integrator/cp目標版.

請參考U-Boot網站 http://www.denx.de/wiki/UBoot

下載檔案u-boot-1.2.0.tar.bz2

下載完解壓縮

# tar jxvf u-boot-1.2.0.tar.bz2

進入目錄後, 還需要修改config檔. 到include/configs目錄

會看到許多版子的設定檔

將下面參數加入設定檔

include/configs/integratorcp.h

#define CONFIG_BOOTFILE     "u-boot.bin"
#define CONFIG_ETHADDR      00:02:f7:00:04:86
#define CONFIG_NETMASK      255.255.255.0
#define CONFIG_IPADDR       192.168.10.1
#define CONFIG_SERVERIP     192.168.10.11

回到目錄下後, 設定目標板核心.

# make CROSS_COMPILE=arm-softfloat-linux-gnu- cp920t_config

# make CROSS_COMPILE=arm-softfloat-linux-gnu-

編譯好之後就將 u-boot.srec檔案用minicom 寫入版子即可..

2007/09/10

GNU toolchain

這裡針對arm 920T版子做設定

主要參考Karim的書, 另外再補充一些東西

1. 設定環境變數:

export PROJECT=embedded
export PRJROOT=/root/${PROJECT}
export TARGET=arm-linux
export PREFIX=${PRJROOT}/tools
export TARGET_PREFIX=${PREFIX}/${TARGET}
export PATH=${PREFIX}/bin:${PATH}
cd $PRJROOT

2. 核心設定

使用linux-2.4.18在下載patch-2.4.18-rmk7.gz patch檔
解壓縮後:
cd linux-2.4.18
gzip -cd ../patch-2.4.18-rmk7.gz patch -p1
make ARCH=arm CROSS_COMPILE=arm-linux- menuconfig

3. binutils

下載binutils-2.10.1.tar.gz
cd ${PRJROOT}/build-tools
tar zxvf binutils-2.10.1.tar.gz
cd build-binutils
../binutils-2.10.1/configure --target=$TARGET --prefix=${PREFIX}
make
make install

4. gcc

下載gcc-2.95.3.tar.gz
patch檔: gcc-2.95.3-2.patch gcc-2.95.3-no_fixinc-1.patch gcc-2.95.3-returntype_fix-1.patch
tar zxvf gcc-2.95.3.tar.gz
cd gcc-2.95.3
patch -p1 < gcc-2.95.3-2.patch
patch -p1 < gcc-2.95.3-no_fixinc-1.patch
patch -p1 < gcc-2.95.3-returntype_fix-1.patch
cd ../build-boot-gcc
../gcc-2.95.3/configure --target=${TARGET} --prefix=${PREFIX} \
> --without-headers --with-newlib --enable-languages=c
make all-gcc
make install-gcc

5. glibc

下載glibc-2.2.3 , glibc-linuxthreads-2.2.3.tar.gz
patch檔: glibc-manual-stdin.texi-1.127.patch

Fixes error./stdio.texi:3269: First argument to cross-reference may not be
empty../stdio.texi:3270: First argument to cross-reference may not be
empty.makeinfo: Removing output file
`/home/dank/downloads/crosstool-0.24/build/mipsel-unknown-linux-gnu/gcc-3.2.3-glibc-2.2.3/glibc-2.2.3/manual/libc.info'
due to errors; use --force to preserve.make[2]: *** [libc.info] Error 2make[2]:
Leaving directory
`/home/dank/downloads/crosstool-0.24/build/mipsel-unknown-linux-gnu/gcc-3.2.3-glibc-2.2.3/glibc-2.2.3/manual'
===================================================================RCS
file: /cvs/glibc/libc/manual/stdio.texi,vretrieving revision 1.126retrieving
revision 1.127diff -u -r1.126 -r1.127--- libc/manual/stdio.texi 2001/06/06
07:11:31 1.126+++ libc/manual/stdio.texi 2001/07/31 18:57:16 1.127@@
-3265,8 +3265,8 @@ If you are trying to read input that doesn't match a
single, fixed pattern, you may be better off using a tool such as Flex to
generate a lexical scanner, or Bison to generate a parser, rather than
using-@code{scanf}. For more
information about these tools, see @ref{, , ,-flex.info, Flex: The Lexical
Scanner Generator}, and @ref{, , ,+@code{scanf}. For more information about
these tools, see @ref{Top, , ,+flex.info, Flex: The Lexical Scanner Generator},
and @ref{Top, , , bison.info, The Bison Reference Manual}.
@node
Input Conversion Syntax

tar -xvzf glibc-2.2.3.tar.gz
tar -xzvf glibc-linuxthreads-2.2.3.tar.gz --directory=glibc-2.2.3
cd build-glibc
CC=arm-linux-gcc ../glibc-2.2.3/configure --host=$TARGET \
> --prefix="/usr"--enable-add-ons \
> --with-headers=$TARGET_PREFIX/include

make
make install_root=$TARGET_PREFIX prefix="" install

6. 修改libc.so

vi ${TARGET_PREFIX}/bin/libc.so

/* GNU ld script Use the shared library, but some functions are only in
the static library, so try that secondarily. */
GROUP ( /lib/libc.so.6 /lib/libc_nonshared.a )
改成
GROUP ( libc.so.6 libc_nonshared.a )

7. 完整gcc的規劃

cd $PRJROOT/build-tools/build-gcc
../gcc-2.95.3/configure --target=$TARGET --prefix=$PREFIX --enable-languages=c,c++
make all
make install

8.配置uClibc
下載uClibc-0.9.16
cd $PRJROOT/build-tools/uClibc-0.2.98
make CROSS=arm-linux- menuconfig

make CROSS=arm-linux-
make CROSS=arm-linux- PREFIX="" install

Zhuang Shang-Yi