Friday, January 28, 2011

How to install subversion on Linux and make it working.

To install svn in linux and make it fully working , I found it really tough and it has taken my lots of time. I am thankful to Sanjay, who helped me in this and without him it won’t be possible.While installing this you will miss a small thing and you got stuck.But this is not at all tough like open source it’s really very easy.

Why I have choosed this topic to write is that, those who are using it first time will need some time to get full of subversion.
Now the latest version of svn is 1.4.6.
To install svn first you need svn tar file. Which can be downloaded easily from here or you can run these commands (depending on your required version).
wget http://subversion.tigris.org/downloads/subversion-1.4.6.tar.gz
wget http://subversion.tigris.org/downloads/subversion_deps-1.4.6.tar.gz (It contains the library required by svn)

This you can download any where, Here I am creating a folder svn inside /usr/local directory.(/usr/local/softwares/svn/).(So it will be easy for me to specify paths and you to understand).
To start the installation process first untar this two file.
tar -xvzf subversion-1.4.6.tar.gz
tar -xvzf subversion_deps-1.4.6.tar.gz
(After untar you won’t find any particular folder named subversion_deps-1.4.6. Don’t worry for that it is keeping all the libraries files inside subversion-1.4.6)
Now have to configure it, but before that we have to check whether our system is having all the dependies or not, like APR(Apache Portable Runtime) and APR utility.
If it’s not there then we can get it from here,
apr-1.2.12.tar.gz
apr-util-1.2.12.tar.gz

once downloaded you can start configuring it.
tar -xvzf apr-1.2.12.tar.gz
cd apr-1.2.12
./buildconf
./configure
make

The apr util directory requires apr directory, so we need to specify path for that.
tar -xvzf apr-util-1.2.12.tar.gz
cd apr-util-1.2.12
./buildconf –with-apr=/path to apr directory (/usr/local/sofwares/svn/apr-1.2.12)
./configure –with-apr=/path to apr directory (/usr/local/sofwares/svn/apr-1.2.12)
make

The other directories we have to check before installation are,
autoconf 2.50 or newer (autoconf –version)
libtool 1.4 or newer (this you can find inside subversion-1.4.6 folder)
Neon Library 0.25.x or 0.26.x (inside subversion-1.4.6 folder)
Berkely DataBase (If it’s not there you have to install it.)
Download the file, db-4.6.21.tar.gz .
Apache web server greater then 2.0.49 (httpd -version)
zlib (inside subversion-1.4.6 folder)
After checking all this, we can start installation process,
cd subversion-1.4.6
./autogen.sh
./configure –with-berkelay-db=/usr/local/softwares/svn/db-4.6.21 (path of berkelay data base)
make
make install

Once we have finished that we need to map svn with apache.
Before that we have to check whether mod_dav_svn and mod_authz_svn modules are present or not,if not then install it.
yum install mod_dav_svn
and add this in /etc/httpd/conf/httpd.conf file
LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule authz_svn_module modules/mod_authz_svn.so

Make sure you will add these lines after LoadModule dav_module modules/mod_dav.so.
Now create one repository with svnadmin command.Run this command inside your subversion folder.
svnadmin create repos
after that you can check with,
ls repos
conf dav db format hooks locks README.txt

This full path you have to specify in SVNPath.
Then at the end of the httpd.conf file you have to add
#for svn path

DAV svn
SVNPath /usr/local/softwares/svn/subversion-1.4.6/repos

now restart ur apache.
For security purpose to give authentication to the user we have to create authentication file with the command,
htpasswd -cm /etc/svn-auth-file user_name (test)
New password : test123
Re-Type passwd : test123

And to specify which access you want to give the user, create svn-access-file inside /etc.
Inside this file add code as,
[/]
* = rw

This will give authentication to every user specified in Require user inside location.
You can specify this inside httpd.conf file as,

DAV svn
SVNPath /usr/local/softwares/svn/subversion-1.4.6/repos
AuthzSVNAccessFile /etc/svn-access-file
AuthType Basic
AuthName “Subversion repository”
AuthUserFile /etc/svn-auth-file
Require user user_name

Now you are ready to import files in the repository, with the following command.
svn import -m “Initial import.” path(from where you want to import file) file:///usr/local/softwares/svn/subversion-1.4.6/repos/(repository path)
after running this command don’t worry it is not goin to display any thing inside repos folder. To check that you can run it in the browser.
http://localhost(you have to give your domain name)/svn/repos
Now you are ready to checkout the files with any svn client. For linux user smartsvn is best.
After doin this you can start ur smartsvn client and can follow these steps,
check out project from repository -> manage -> add -> enter svn url
enter ur url here http://localhost/svn/repos
enter ->ok
server name and repository path it will take automatically. click next, choose user radio button and give user name and password.
follow the steps to fetch the files.
To install smartsvn, first get the smartsvn tar file.
untar the file, with the command
tar -xvzf smartsvn-version.gz
for smartsvn first check the correct version of java is available or not, with
java -version command.If it is coming like this then you can proceed.
java version “1.5.0_14″
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) Server VM (build 1.5.0_14-b03, mixed mode)
Sometimes java will be installed but not the JRE. In that case simply re-install java.
Can install the latest version of java with rpm bin file.
jdk-1_5_0_14-linux-i586-rpm.bin
./jdk-1_5_0_14-linux-i586-rpm.bin

go through the license and say yes.
It will installed in /usr/java/
Now have to make chnages in .bash_profile file.
cd /root
vi .bash_profile

give JAVA_HOME path.
JAVA_HOME = /usr/java/jdk1.5.0_14
Then compile it with the command,
source .bash_profile
then again check with java version.
Now you are ready to utilise the full feature of subversion.

Creating Users and Setting Permissions in MySQL

Creating Users and Setting Permissions in MySQL - Opening the MySQL console application



There are two major ways to run the MySQL console. You can either sit down at the physical server, or you can telnet into the server and run the application remotely. In either situation, you will need to be at the command prompt of the machine that is running the MySQL server.

The next most logical step is to determine the directory where MySQL is installed. On most Unix and Linux based installations, MySQL is located in the /usr/local/bin directory. On a Windows server, MySQL is most likely found under c:/mysql/. If you are having difficulty locating MySQL, check with your installation’s documentation.

To create users and set permissions, you have to run the console as a user that has permission to do so. By default, MySQL is installed with a root account with these permissions. Typically this account has no password when connecting from the localhost.

To start the MySQL console, at the command prompt, change to the MySQL directory and run the console by typing "mysql –u=root". This opens the MySQL console as the user root. If all goes well, you will be greeted by something similar to the following prompt:

Welcome to the mysql monitor. Commands end with ; or \g.
Your mysql connection id is 4 to server version 3.23.36

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.

mysql>


It is not always necessary to specify the user as root, however, it has been included here to help ensure the proper connection to the MySQL console. If you are having trouble or a root password has also been specified you can try to launch mysql with the following revised command: "mysql –u=root –p=passwrd". Keep in mind that you should replace passwrd with your actual root password.

Setting up a new user
Now we are ready to setup some MySQL user accounts. Let's run through a few GRANT statements and discuss in detail what they are doing. These statements should be entered in at the MySQL console prompt. Be sure to include all of the quotations in the statements and to end each statement with a semicolon.

mysql> create database somedb
-> grant all privileges on somedb.* to someusr@"localhost" identified by 'passwrd';


The first part of this statement creates a database called somedb. This step can be left out if the database that you want to use already exists. The second part is the actual statement that sets up the user and the privileges held by that user. Lets look at it in greater detail.

somedb.*

This specifies the database for which the user created will be allowed to access. The .* indicates that this user will be permitted to work with all of the tables within the database somedb. If you are interested in limiting the user to only one table, then the name of that table should be specified after the period in place of the asterisk.

someusr@"localhost"

This is what actually sets up the user information. The someusr is the username that is being created. The portion after the @ indicates the host from which this user is allowed to connect. In this case, this user can only connect from the localhost.

passwrd

This is the password to be used by the user. It is suggested that you use something more creative than the one indicated here.

Connecting from any host
In the previous example, we created a user that could only connect to a specific database from the local machine. In this example, we loosen things up a bit and allow the user a little more freedom. Here is the statement:

mysql> grant all privileges on *.* to someusr@"%"
> identified by 'passwrd';


As with the previous example, lets take a look at the statement piece by piece.

*.*

This indicates that the user will be permitted to connect to all of the MySQL databases and all of the tables contained in those databases.

someusr@"%"

The user is again specified as someusr, however, this time a host of "%" is used. This indicates that this user may connect to the database from any host or IP number. A more secure way to do this is to specify the IP number of your workstation, however, sometimes this is not possible depending on your network setup.

As in the previous example, the password is set as passwrd.


Creating Users and Setting Permissions in MySQL - Reducing User Privileges




In both of the previous examples, we created users that were granted all privileges on their respective database(s). It may be wise to consider reducing these privileges, especially if the user is connecting from a remote workstation. This is accomplished with the following statement:

mysql> grant select,insert,update,delete,create,drop
-> on somedb.* to someusr@"%" identified by 'passwrd';


With this statement, the user someusr is allowed limited access to the database somedb from any host ("%") using the password passwrd.

The GRANT statement specifies that this user is only allowed to run a limited amount of statements on the MySQL server. This user will be allowed to: select records, insert records, update records, delete records, create databases, and DROP DATABASEs. More importantly, this user is not permitted to create users and set privileges.

You could further reduce a user's privileges by removing other items from the GRANT statement. The best policy here is that the user should only be given permission to access the functions that are necessary to perform their tasks.

Refresh the MySQL grant tables
After going through and setting up users and permissions with grant statements, there is one final step to making the whole thing work.

At this point it is important for you to understand where all of this permission information is kept. When MySQL is installed, a database called mysql is automatically created. This database contains several tables that hold all of the information pertaining to each user, the databases they have access to, the hosts they can connect from, and the privileges allowed for each.

MySQL only loads these user tables and the permissions held within when it first boots. It does not take another look at those tables unless it is told to do so. So, without this step, none of your newly created users will work at all.

There are several ways to reload the privilege tables, however since we are already logged in to the MySQL console, we can do so by running the following command:

mysql> flush privileges;

This tells MySQL to take another look at the user tables and hence puts all of your new users and privileges into operation.




How to Change Mysql Data Directory


Let’s say your second HDD is mounted as /home2

1. The first step is to stop MySQL so that all your data gets copied correctly.
$ /etc/rc.d/init.d/mysql stop
- or -
$ service mysql* stop

2. Create the new database directory in the second HDD and for this let’s say it’s named as mysqldata
$ mkdir /home2/mysqldata

3. Copy the database files from the first HDD to the second HDD
$ cp -R /var/lib/mysql/ /home2/mysqldata

4. Set the correct owner and group, permissions of the new database directory on the 2nd HDD
$ chown -R mysql.mysql /home2/mysqldata/

5. Rename your old database directory
$ mv /var/lib/mysql/ /var/lib/mysql_old

6. Create a symbolic link from the old database directory to the new one for any programs that rely on the default location
$ ln -s /home2/mysqldata/ /var/lib/mysql

7. Set the correct owner and group on the symbolic link
$ chown mysql.mysql /var/lib/mysql

8. Edit the configuration file (/etc/my.cnf) to update the changes
Comment out the old settings and add a line for the new one as you can see below
[mysqld]
#datadir=/var/lib/mysql
datadir=/home2/mysqldata
#socket=/var/lib/mysql/mysql.sock
socket=/home2/mysqldata/mysql.sock
#basedir=/var/lib
basedir=/home2
save my.cnf and exit your text editor
Note that for MySQL version 5 you have to remove the line basedir. The basedir line is only for those who are using MySQL version 4.

9. Restart MySQL
$ /etc/rc.d/init.d/mysql start
- or -
$ service mysql* start

10. If MySQL refuses to start look in /var/log/mysqld.log for the reason

What is Mysql Data Directory?

Mysql data directory is important part where all the mysql databases storage location.By default MySQL data default directory located in /var/lib/mysql.If you are running out of space in /var partition you need to move this to some other location.

Note:- This is only for advanced users and before moving default directory make a backup of your mysal databases.

Procedure to follow
Open the terminal
First you need to Stop MySQL using the following command
/etc/init.d/mysql stop
Now Copy the existing data directory (default located in /var/lib/mysql) using the following command
cp -R -p /var/lib/mysql /path/to/new/datadir
All you need are the data files, so delete the others with the command
rm /path/to/new/datadir
Note:- You will get a message about not being able to delete some directories, but that’s what you want.
Now edit the MySQL configuration file with the following command
Vim /etc/mysql/my.cnf
Look for the entry for “datadir”, and change the path (which should be “/var/lib/mysql”) to the new data directory.

you’ll never be able to restart MySQL with the new datadir location.
In the terminal, enter the command
vim /etc/apparmor.d/usr.sbin.mysqld
Copy the lines beginning with “/var/lib/mysql”, comment out the originals with hash marks (“#”), and paste the lines below the originals.
Now change “/var/lib/mysql” in the two new lines with “/path/to/new/datadir”. Save and close the file.
Restart the AppArmor profiles with the command
 /etc/init.d/apparmor reload
Restart MySQL with the command
 /etc/init.d/mysql restart
Now MySQL should start with no errors, and your data will be stored in the new data directory location.

Thursday, January 27, 2011

Linux Basics

       What is Linux?
    Linux is an Open Source, free Operating System created by Linus Torvalds. For more information, please see our General Information page.
       What is the linux kernel?
    The Linux kernel is the core of the Linux operating system. To find out which version of the Linux kernel you are running, simply type uname -r in a terminal. For more information on the Linux kernel, please visit http://www.kernel.org. For information on how to install or compile the Linux Kernel, please see our Kernel Compile/Upgrade Guide.
       What distribution of Linux should I install?
    One of the pleasures of running linux is choice. The choice of which Linux Distribution you wish to run is 100% up to you. You can browse our Distributions Page for comparisons of the various distributions out there. If you are totally new to linux we suggest you start off with either Red Hat Linux or Mandrake Linux as they are easier to install, configure and maintain.
       Who is the root user?
    The root user is the master user or systems adminstrator of the machine. The root user has full access to all files and directories, which is why you will only want to use the root account to perform system maintainance or to install applications.
       How do I reboot my machine?
    In order to reboot your machine, you must instruct the Linux kernel to stop all the processes (programs) it's currently running. Do not simply press the reboot button on the front of your computer case, as Linux must be properly shut down/restarted. To restart your machine, as root, run either of the following commands in a terminal: reboot shutdown -r now On most systems you can also press (all at once) the control, alt and delete keys.
       How do I shut down my machine?
    In order to shut down your machine, you must instruct the Linux kernel to stop all the processes (programs) it's currently running. Do not simply press the power off button on the front of your computer case, as Linux must be properly shut down/restarted. To shut down your machine, as root, run either of the following commands in a terminal: halt shutdown -h now
       How do I remove/uninstall Lilo/Grub?
    If you want to remove the Lilo or Grub boot loader from your system (if you are uninstalling linux etc) follow the procedures below: For Windows 95/98 etc: Use a dos boot disk and at the command prompt type: fdisk /mbr For Windows XP: Boot off the XP Installation CD and go into rescue mode. From there run the command: fixmbr
    How do I add a user?
    While logged in as root, type adduser username at the shell prompt, replacing username with the name you want to use to log in. You can create as many users as you wish, one for each member of your family, for example.
    How do I delete a user?
    While logged in as root, type userdel username at the shell prompt, replacing username with the user name you wish to delete.
    How do I change my login password?
    To set the password for a specific user, type (while logged in as root) passwd username at the shell prompt, replacing username with the name of the person who's password you wish to change. By typing just passwd you'll be able to change the password of the user under who's name you're currently logged in as. Do not use passwords which are easy to guess, like your middle name or your pets name.
    How do I change directories?
    cd - Used to change directories Usage: cd [directory] Example: cd /home/joey/html Example: cd ../ (to move back one directory) Example: cd (to return to your home directory) For more information, in a terminal, type man cd
    How do I copy files?
    cp - Used to copy files/directories from one location to another Usage: cp file newlocation Example: cp /home/joey/index.html /var/www/index.html Example: cp /home/joey/* /var/www/ (this will copy everything in /home/joey to /var/www/) For more information, in a terminal, type man cp
    How do I rename files?
    mv - Used to move or rename files Usage: mv file location (to move) Usage: mv filename newfilename (to rename) Example: mv index.html /var/www/index.html (to move) Example: mv index.html index2.html (rename) For more information, in a terminal, type man mv
    How do I move files?
    mv - Used to move or rename files Usage: mv file location (to move) Usage: mv filename newfilename (to rename) Example: mv index.html /var/www/index.html (to move) Example: mv index.html index2.html (rename) For more information, in a terminal, type man mv
    How do I edit files
    The default editor that ships with almost every distribution of Linux is Vi. This editor is slightly advanced and confuses most newbies. We suggest starting off with Pico, a more simple editor. Pico is part of the Pine software package which can be downloaded at http://www.washington.edu/pine (It might already be installed on your system). To edit a file with pico, simply run: pico filename This will open up the file and allow to you edit it. At the bottom of the screen you will notice the various commands that you have access to with Pico. If you wish to save the file and exit, simply hold down the control key and hit the x key on your keyboard.
    How do I list directory contents?
    ls - To list the contents of a directory Usage: ls [flags] directory Example: ls (To list the current directory) Example: ls /home/joey (To list the contents of /home/joey) Example: ls -a (To list hidden files) Example: ls -l (To list file/directory permissions and file sizes) Example: ls -al /home/joey (To list all files and permissions in /home/joey) For more information, in a terminal, type man ls
    How do I delete files?
    To delete a file you must first have write permission to it. For information about permissions, click here. Once you have write permission, in a terminal run: rm filename There is no "Recycle Bin" in Linux so once you delete a file, it's gone for good. When removing files, you may use an astrix (*) as a wildcard flag to remove certain files, for example if I wanted to remove all files that began with the letter j, I would run rm j* If anyone tells you to run rm -rf / as root, DO NOT LISTEN TO THEM. Running this command will delete all the files/directories on your Linux system.
    How do I delete directories?
    If you have ownership to the directory and the directory is empty, you can simply type rmdir directoryname to remove the directory. If the directory is not empty and you wish to simply delete it and all its contents, run rm -rf directoryname Please be careful with the -rf flag, as it will remove everything in the specified directory including sub directories. With root access and the rm -rf command you can wipe out your entire system if you make an error.
    How do I access my cdrom drive?
    Linux requires you to mount your cdrom/floppy drives when you wish to use them. On most Linux distributions, the mount command will require root access. Depending on which Linux distribution you run, one of the following commands should mount your cdrom drive. As root, run: mount /dev/cdrom /mnt/cdrom mount /dev/cdrom /cdrom
    How do I access my floppy drive?
    Linux requires you to mount your cdrom/floppy drives when you wish to use them. On most Linux distributions, the mount command will require root access. Depending on which Linux distribution you run, one of the following commands should mount your floppy drive. As root, run: mount /dev/fd0 /mnt/floppy mount /dev/fd0 /floppy
    How do I view/change ownership on files/directories?
    One of the great features of Linux is that it is a multi-user system. With multi-users, it allows certain users to own files and directories so nobody else can access/modify them, for example your /home directory. To change the ownership of a file/directory, as root execute the following command in a terminal: chown username. filename/directory This will change the ownership and group ownership of the specified file or directory to the specified user. For more information, in a terminal, type man chown
    How do I view/change permissions on files/directories?
    There are 3 different attributes a file can have that make up the permissions for the file. There is read access that allows users to read the file. There is write access that allows users to modify the file. There is executable access that allows users to execute the file.
    From these three attributes, you now have three different levels of permissions, the first being for the owner of the file, the second being for the group the file belongs to and the third being all other users on the system.
    To view the current permissions on files and directories run the following command: ls -l and you should see something that looks like:
    drwxrwxr-x    3 joey     html         4096 Sep 12  2000 images/
    -rw-rw-r--    1 joey     html          267 Aug  8 12:55 index.shtml
    
    The above shows you that images is a directory (d) and that the owner and group (joey html) have read (r), write (w) and execute (x) permissions while other users on the system only have read (r) and execute (x) permissions. For the index.shtml file, it shows joey and html as the owner and group and they both have read and write access to the file. It also shows that other users on the system only have read access. Note there is no executable permission since the file is not an executable.
    To modify the permissions on a file, you either have to own it or be logged in as root. To modify the permissions, in a terminal type: chmod xxx filename/directory You will have to replace the xxx flags with the permissions you wish to change. You can either go by the numerical value or by the actuals. Some common numerical values are: 755 - Read, Write, Execute for owner, read, execute for group and other. 644 - Read, Write for owner, read for group and other. 666 - Read, Write for all. 700 - Read, Write, Execute for owner, nothing for group, other. So if you wanted to make a file an executable for your user and other users on the system, you would run the following: chmod 755 somefile For more information, in a terminal, type man chmod
    How do I find files on my system?
    There are two methods to search for files on your Linux machine, one method being the locate command and the other being the find command. If you wish to use the locate command, you must first update the locate database by running the following command in a terminal as root: /usr/bin/updatedb This will create an index of all the files and their locations on your hard drive. Once updatedb has completed running, you may now search your drive for a specific file by running the following command in a terminal: /usr/bin/locate filename
    Another way to search your linux system is with the find command. To find a file on your sytem with the find command, in a terminal run the follwoing: /usr/bin/find / -name filename Be sure to replace "filename" with the actual name of the file.
    How do I unzip a .tar.gz/.tgz file?
    To extract .tar.gz or .tgz files, run the following command in a terminal: tar -zxvf file.tar.gz (or file.tgz) This will normally create a new directory based on the filename. If you want to extract a filename called file.tar (without the .gz) simply run: tar -xvf file.tar For more information, in a terminal, type man tar
    How do I unzip a .bz2 file?
    To extract .bz2 files, run the following command in a terminal: bunzip2 -dv file.bz2 This will normally create a new directory based on the filename. For more information, in a terminal, type man buznip2
    How do I install a program?
    There are many ways to install applications in Linux. If the file you wish to install is in .rpm format and you are running an RPM based distribution such as Red Hat, Mandrake or SuSE, run the following command as root in a shell prompt/terminal: rpm -Uvh filename.rpm
    If you are running Debian GNU/Linux and you wish to install a .deb file, you can do so by running the following command in a terminal as root: dpkg -i filename.deb or apt-get install package (if you have APT already installed/configured).
    If the file you with to install is in a .tar.gz, .tgz or .bz2 format, you will first have to decompress (unzip) the file. Once the file has been unziped, it should create a directory based on the filename. Simply change into that directory and open up the README or INSTALL file in a text editor and follow the installation instructions.
    How do I run a program?
    To run a program, you must first make sure it has executable permissions set. For more information on permissions, see How do I view/change permissions on files/directories? above. Once the file has executable permissions, simply run the filename from a terminal. I will use the filename "irc" as an example Example: /usr/bin/irc (from anywhere on the system) Example: ./irc (from the /usr/bin directory)

Wednesday, January 26, 2011

Backing Up and Restoring Your MySQL Database

Backing Up and Restoring Your MySQL Database


MySQL 5.5 News: Read the MySQL 5.5 Update feature overview.

The easiest way to backup your database would be to telnet to the your database server machine and use the mysqldump command to dump your whole database to a backup file. If you do not have telnet or shell access to your server, don't worry about it; I shall outline a method of doing so using the PHPMyAdmin web interface, which you can setup on any web server which executes PHP scripts.

Playing with mysqldump

If you have either a shell or telnet access to your database server, you can backup the database using mysqldump. By default, the output of the command will dump the contents of the database in SQL statements to your console. This output can then be piped or redirected to any location you want. If you plan to backup your database, you can pipe the output to a sql file, which will contain the SQL statements to recreate and populate the database tables when you wish to restore your database. There are more adventurous ways to use the output of mysqldump.

A Simple Database Backup:

You can use mysqldump to create a simple backup of your database using the following syntax.

mysqldump -u [username] -p [password] [databasename] > [backupfile.sql]

o [username] - this is your database username
o [password] - this is the password for your database
o [databasename] - the name of your database
o [backupfile.sql] - the file to which the backup should be written.


The resultant dump file will contain all the SQL statements needed to create the table and populate the table in a new database server. To backup your database 'Customers' with the username 'sadmin' and password 'pass21' to a file custback.sql, you would issue the command:

mysqldump -u sadmin -p pass21 Customers > custback.sql

You can also ask mysqldump to add a drop table command before every create command by using the option --add-drop-table. This option is useful if you would like to create a backup file which can rewrite an existing database without having to delete the older database manually first.

mysqldump --add-drop-table -u sadmin -p pass21 Customers > custback.sql

Backing up only specified tables

If you'd like restrict the backup to only certain tables of your database, you can also specify the tables you want to backup. Let's say that you want to backup only customer_master & customer_details from the Customers database, you do that by issuing

mysqldump --add-drop-table -u sadmin -p pass21 Customers customer_master customer_details> custback.sql

So the syntax for the command to issue is:

mysqldump -u [username] -p [password] [databasename] [table1 table2 ....]

o [tables] - This is a list of tables to backup. Each table is separated by a space.

Backing Up and Restoring Your MySQL Database - What about Multiple Databases?


If you are a database administrator who has to look after multiple databases, you'll need to back up more than one database at a time. Here's how you can backup multiple databases in one shot.

If you want to specify the databases to backup, you can use the --databases parameter followed by the list of databases you would like to backup. Each database name has to be separated by at least one space when you type in the command. So if you have to backup 3 databases, let say Customers, Orders and Comments, you can issue the following command to back them up. Make sure the username you specify has permissions to access the databases you would like to backup.

mysqldump -u root -p pass21 --databases Customers Orders Comments > multibackup.sql

This is okay if you have a small set of databases you want to backup. Now how about backing up all the databases in the server? That's an easy one, just use the --all-databases parameter to backup all the databases in the server in one step.

mysqldump --all-databases> alldatabases.sql

Backing up only the Database Structure

Most developers need to backup only the database structure to while they are developing their applications. You can backup only the database structure by telling mysqldump not to back up the data. You can do this by using the --no-data parameter when you call mysqldump.

mysqldump --no-data --databases Customers Orders Comments > structurebackup.sql

Compressing your Backup file on the Fly

Backups of databases take up a lot of space. You can compress the output of mysqldump to save valuable space while you're backing up your databases. Since mysqldump sends its output to the console, we can pipe the output through gzip or bzip2 and send the compressed dump to the backup file. Here's how you would do that with bzip2 and gzip respectively.

mysqldump --all-databases | bzip2 -c >databasebackup.sql.bz2

mysqldump --all-databases | gzip >databasebackup.sql.gz

A Shell Script for Automating Backups?

You can automate the backup process by making a small shell script which will create a daily backup file. How do you get cron to back up your database without overwriting the older backup? You can use a tiny shell script to add the date to your backup file. An example of a shell script you could use is shown below.

#!/bin/sh
date=`date -I`
mysqldump --all-databases | gzip > /var/backup/backup-$date.sql.gz


Backing Up and Restoring Your MySQL Database - Easy Restore


Now that you've got backups of your database, let's learn how to restore your backup in case your database goes down. Here's how you can restore your backed up database using the mysql command.

Restore using mysql

If you have to re-build your database from scratch, you can easily restore the mysqldump file by using the mysql command. This method is usually used to recreate or rebuild the database from scratch.

Here's how you would restore your custback.sql file to the Customers database.

mysql -u sadmin -p pass21 Customers < custback.sql

Easy isn't it ? Here's the general format you would follow:

mysql -u [username] -p [password] [database_to_restore] < [backupfile]

Now how about those zipped files? You can restore your zipped backup files by first uncompressing its contents and then sending it to mysql.

gunzip < custback.sql.sql.gz | mysql -u sadmin -p pass21 Customers

You can also combine two or more backup files to restore at the same time, using the cat command. Here's how you can do that.

cat backup1.sql backup.sql | mysql -u sadmin -p pass21

Moving Data Directly Between Databases

How would you like to replicate your present database to a new location? When you are shifting web hosts or database servers, you can directly copy data to the new database without having to create a database backup on your machine and restoring the same on the new server. mysql allows you to connect to a remote database server to run sql commands. Using this feature, we can pipe the output from mysqldump and ask mysql to connect to the remote database server to populate the new database. Let's say we want to recreate the Customers database on a new database server located at 202.32.12.32, we can run the following set of commands to replicate the present database at the new server.

mysqldump -u sadmin -p pass21 Customers | mysql --host=202.32.12.32 -C Customers


PHPMyAdmin is an easy to use web-interface to control MySQL databases. You can run this from any system which has a web server which runs PHP scripts. PHPMyAdmin also lets you administer multiple database servers using the same interface. If you don't have telnet or shell access to your database server, this is probably how you would be administering your databases. If you haven't used PHPMyAdmin, head over to http://www.phpmyadmin.net/ and download the latest version to experiment with.

Configure PHPMyAdmin

Before I go on to show you how you can back up and restore your databases using PHPMyAdmin, let me take a little while to show those who going to be first time users of PHPMyAdmin how to configure their installation. Once you've downloaded the latest version of PHPMyAdmin, copy the files from the installation source to your web server. Before you run the program, you'll have to edit a few lines in config.inc.php to get your installation to work properly. You can find this file in the root folder of the installation.

Head over to the line which reads:

$cfg['PmaAbsoluteUri'] = '';

Change this to the web URL of the installation. Let's say you've copied the files to the directory 'phpmyadmin' under the URL 'http://dbserver.com', you should change the option to:

$cfg['PmaAbsoluteUri'] = 'http://dbserver.com/phpmyadmin/';

Here are some more options about your database server which you'll have to modify before you save the configuration file. Let's say you have to connect to your database server, which is residing at 202.32.12.2 and connects on port 432, with the username 'jester' and password 'freddy32', the options you have to change will look like this:

$cfg['Servers'][$i]['host'] = '202.32.12.2';
$cfg['Servers'][$i]['port'] = '432';
$cfg['Servers'][$i]['connect_type'] = 'tcp';
$cfg['Servers'][$i]['auth_type'] = 'config';
$cfg['Servers'][$i]['user'] = 'jester';
$cfg['Servers'][$i]['password'] = 'freddy32';

Once you've got the correct setting for the database entered in these options, you can save your configuration file and head over to your PHPMyAdmin installation through your browser.

Backing Up and Restoring Your MySQL Database - Backing Up and Restoring Your Database with PHPMyAdmin


You can backup your database using PHPMyAdmin in just a few mouse clicks. First head over to your database by clicking the database name in the list on the left of the screen. Once you get to your database details, you should get a menu header for your database which looks like so:

Back up with PHPMyAdmin

Click on Export. This will get you to a screen with the following options.

Backup with PHPMyAdmin

From here it's just a matter of clicking the right options and the 'Go' Button to get your database dump. To save the backup file on your computer, click on the Save as file option and the corresponding compression option to get the backup to download to your system.

Restoring your Database via PHPMyAdmin

Restoring your database is as easy as backing it up. If you would like to rewrite the backup over an existing database, click on the database name in the database list on the left, click all the check boxes next to the table names and select Drop in the With selected: drop down box.

Restore with PHPMyAdmin

This will drop all existing table in the database. Then head over to the top menu bar and click on SQL. This will bring up a window where you can either type in SQL commands, or upload your SQL file. The window should look like this:

Restore with PHPMyAdmin

Click on the Browse button, and select the sql backup file you wish to upload to the server. Once you've selected the file, click on 'Go'. This will upload the backup file to the server and execute the SQL commands in the backup and restore your database.

Now that you know how easy it is to backup and restore your database, spend a few minutes everyday to backup your important data. You can even automate the backup process by using a task scheduler like cron.

Thursday, January 6, 2011

How email works

Email, as simple as it is to use, relies on a more complicated set of operating procedures than that of the Web. For most users, its operation is transparent, which means that it is not necessary to understand how email works in order to be able to use it.

However, the short introduction below has been provided to help users understand its basic principles, give them an idea of how to best configure their email clients and inform them about the underlying mechanisms of spam.

How email works

Email is based around the use of electronic mailboxes. When an email is sent, the message is routed from server to server, all the way to the recipient's email server. More precisely, the message is sent to the mail server tasked with transporting emails (called the MTA, for Mail Transport Agent) to the recipient's MTA. On the Internet, MTAs communicate with one another using the protocol SMTP, and so are logically called SMTP servers (or sometimes outgoing mail servers).

The recipient's MTA then delivers the email to the incoming mail server (called the MDA, for Mail Delivery Agent), which stores the email as it waits for the user to accept it. There are two main protocols used for retrieving email on an MDA:

  • POP3 (Post Office Protocol), the older of the two, which is used for retrieving email and, in certain cases, leaving a copy of it on the server.
  • IMAP (Internet Message Access Protocol), which is used for coordinating the status of emails (read, deleted, moved) across multiple email clients. With IMAP, a copy of every message is saved on the server, so that this synchronisation task can be completed.

For this reason, incoming mail servers are called POP servers or IMAP servers, depending on which protocol is used.

Schematic diagram of MTA-MDA-MUA

To use a real-world analogy, MTAs act as the post office (the sorting area and mail carrier, which handle message transportation), while MDAs act as mailboxes, which store messages (as much as their volume will allow) until the recipients check the box. This means that it is not necessary for recipients to be connected in order for them to be sent email.

To keep everyone from checking other users' emails, MDA is protected by a user name called a login and by a password.

Retrieving mail is done using a software program called an MUA (Mail User Agent).

When the MUA is a program installed on the user's system, it is called an email client (such as Mozilla Thunderbird, Microsoft Outlook, Eudora Mail, Incredimail or Lotus Notes).

When it is a web interface used for interacting with the incoming mail server, it is called webmail.

Open relay

By default, and for historical reasons, it is not necessary to authenticate oneself to send email, which means that it is very easy to falsify one's own address when sending mail.

For this reason, nearly all Internet service providers lock down their SMTP servers to that only their subscribers can use them, or more precisely only machines whose IP address belongs to the ISP's domain. This explains why travelling users must modify the outgoing server settings in their email clients each time they move to a new home or business.

When an organisation's email server is improperly configured and allows third-party users on any network to send emails, this is called an open relay.

Open relays are generally used by spammers, as using them hides the true origins of their messages. As a result, many ISPs keep an up-to-date blacklist of open relays, to keep subscribers from receiving messages from such servers.

How DNS Works



1. The user enters asks Netscape to go to "http://www.woz.org" and hits return.

[User Opens Location

2. The computer's TCP stack doesn't know what address "www.woz.org" points to, so it calls upon its DNS server (192.100.81.101) for the address.

[User's TCP Stack makes request to set name server]

3. The DNS server runs the zone "netcom.net" and doesn't handle "woz.org". It first looks in its cache to see if its looked it up before, if so it just returns the address. Unfortunately the server hasn't looked up "netcom.net" before (or its cache entry has timed out), so it queries the server above it ".com" name server at the InterNIC (Internet Information Center) for the "woz.org" server.

[NOTE]

Cached lookups on a domain name server are given "time-out values." This rids us of the problem of old entries being passed around. Time-out values are usually a minutes (for often-changed names) to more than a week. Time-out values are set by the person who runs the name server for a zone. This means that the administrator of "woz.org" can only set the time-out values for "woz.org" entries, and cannot modify "netcom.net" or "apple.com" entries, etc.


[NOTE]

The InterNIC is where everybody must register their domain names. It keeps the hierarchy in tact so it works. It also houses what is called the "root servers," which point to ".org", ".net", ".com", etc. points in the Domain Name Space hierarchy.

[set server queries root servers]

4. The root servers pass the request to the ".org" root server.

[root servers query

5. The ".org" root server passes looks up the "woz.org" server and finds it, so it passes the request to "woz.org"'s name server.

[NOTE]

Note that the root servers only look up where "woz.org" is. They are not responsible for any of the children ("www", "dns", etc.) that "woz.org" is authoritative for. In this case, the root servers are authoritative for ".org", ".com", etc. They delegate authority for other domains to their children in the domain name space tree. Here is what the tree would look like going to the "woz.org" domain:

[DNS Tree for

Note that the root servers are "parents" to ".org", which is the "parent" of anything under it, including "woz", which is the "parent" of "dns", "www" and anything else that ends with "woz.org".

6. The "woz.org" name server looks in its table for a "www" entry. It finds it, and returns its address, 207.142.33.2.

[

7. The request goes back to the sender, who's address has been retained the entire time as the originator of the name query.

[set server receives answer]

8. The address 207.142.33.2 is added to the "netcom.net" name server's cache with a 1 day time-out, which means that it doesn't have to take the above steps again for an entire day.

[user's TCP stack is told the address of the domain name]

9. The "netcom.net" name server returns the address to the user's TCP stack, which tells Netscape what address on the Internet to connect to (see above diagram).

10. The user gets connected to "www.woz.org".

As you can see, the entire process is very structured, which lends to the reason DNS has been so widely accepted and is now the standard for name/address resolution on the Internet.

There is another type of lookup called a "reverse lookup" which looks for the name of a server, instead of its address. So, knowing its address, it queries the name server for the name of the server using the same process you just saw. This is also known as an "IN-ARPA" lookup.

How Name Servers Get Their Data

Parent name servers have to be able to query their siblings for data. This is called a "Zone Transfer" and is a special type of request. Zone transfers are used for when a zone's DNS server goes down. For example, if "ns1.apple.com" does zone transfers with "ns2.apple.com" we can be sure if "ns1" goes down that "ns2" will handle DNS queries until "ns1" is fixed (assuming "ns2" isn't down as well). How do servers know to switch? When you set up your domain with the InterNIC you specify a secondary name server, that's how! Isn't this structured approach great?

When the zone transfer request is sent by a parent server the name server sends a complete zone list in response and increments the zone's "serial number". If the parent server's last serial number from a zone transfer is less than the one in the transferred zone file then it needs to update its entries. If the number is the same then no updates have been made and the zone transfer data isn't used.