AIX > Administrator > Systems Management

Data Versus Metadata for AIX

Data Versus Metadata

Many people are confused by the concept of data and metadata. Though both are different forms of data, they have different uses and different specifications. Data can simply be a piece of information, a list of measurements or observations, a story or a description of a certain thing. Metadata specifies the relevant information about the data, which helps in identifying the nature and feature of the data.

In this article I have scenarios to prove that data can be restored even if we delete metadata information represented in the filesystems information, primary blocks, ODM or VGDA.
 
1. Checking volume group and filesystem information:
 
# lsvg -l testvg
testvg:
LV NAME  TYPE   LPs  PPs  PVs  LV STATE   MOUNT POINT
loglv00  jfs2log 1   1     1    open/syncdN/A
fslv00   jfs2   64   64    1   open/syncd /testfs
 
# df -g
...
/dev/fslv00      1.00     1.001%   4    1%   /testfs
...
 

2. Checking content files information and i-node numbers:
 
# pwd
/testfs
# ls -li
3 -rw-r--r--    1 root    system           34 Oct  9 00:48 file1
4 -rw-r--r--    1 root    system           35 Oct  9 01:45 file2
 
# cat file1
hello,
This is my test fileeeeeee
 
# cat file2
hello, this is my second file!
EOF

 
3. Grabbing fileplace information for specific file(s):
 
# fileplace -pv /testfs/file1
File: /testfs/file1  Size: 34 bytes  Vol: /dev/fslv00
Blk Size: 4096  Frag Size: 4096  Nfrags: 1
Inode: 3  Mode: -rw-r--r--  Owner: root Group: system
Physical Addresses (mirror copy 1)              Logical Extent
----------------------------------------------------------------
00528991       hdisk1 1 frags 4096 Bytes,  100.0%    00000063
 
1 frags over space of 1 frags:   space efficiency = 100.0%
1 extent out of 1 possible:   sequentiality = 100.0%
 
# fileplace -pv /testfs/file2
File: /testfs/file2  Size: 35 bytes  Vol: /dev/fslv00
Blk Size: 4096  Frag Size: 4096  Nfrags: 1
Inode: 4  Mode: -rw-r--r--  Owner: root Group: system
 
Physical Addresses (mirror copy 1)              Logical Extent
----------------------------------------------------------------
00528992       hdisk1 1 frags 4096 Bytes,  100.0%    00000064

 
4. Unmounting the filesystem only, assuming the volume group is
Fine, but the filesystem is corrupted and/or cannor be mounted:
 
# cd ; umount /testfs
 
5. Checking filesystem dump/tree information (Assuming we cannot
Mount it anyway:
 
# fsdb /testfs
File System:                    /testfs
File System Size:               2096680 (512 byte blocks)
Aggregate Block Size:           4096
Allocation Group Size:          8192    (aggregate blocks)
 
- We will get the fsdb prompt ">", and we will work based on the i-node numbers we got in setp_2
- For the file file1, it has inum 3 - I will use it in the fsdb
Prompt as below:
> i 3
Inode 3 at block 57, offset 0x600:
 
[1] di_fileset:         16              [18] di_inostamp:       0x5bbc3ff3
[2] di_number:          3               [19] di_gen:            1083267616
[3] di_size:    0x0000000000000022      [20] di_ixpxd.len:      4
[4] di_nblocks: 0x0000000000000001      [21] di_ixpxd.addr1:    0x00
[5] di_nlink:           1               [22] di_ixpxd.addr2:    0x00000039
[6] di_mode:            0x000081a4           di_ixpxd.address:  57
                0100644 -rw-r--r--      [24] di_uid:            0
                                        [25] di_gid:            0
[9] di_atime.tj_nsec:   0x1db14452      [26] di_atime.tj_sec:0x000000005bbc4e96
[10] di_ctime.tj_nsec:  0x3928b5b9      [27] di_ctime.tj_sec:0x000000005bbc413a
[11] di_mtime.tj_nsec:  0x3928b5b9      [28] di_mtime.tj_sec:0x000000005bbc413a
[12] di_otime.tj_nsec:  0x3928b5b8      [29] di_otime.tj_sec:0x000000005bbc413a
[13] di_ea.flag:        0x00            [30] di_ea.len:         0
     EAv1                               [31] di_ea.addr1:       0x00
[15] di_ea.nEntry:      0x00            [32] di_ea.addr2:       0x00000000
[16] di_ea.type:        0x0000               di_ea.address:     0
                                        [34] di_ea.nblocks:     0
change_inode: [m]odify, [e]a, [t]ree, or e[x]it >
 
- Then I will use the "t" subcommand, which is the tree:
change_inode: [m]odify, [e]a, [t]ree, or e[x]it > t
[1] flag        0x83            BT_ROOT  BT_LEAF
[2] nextindex   3
[3] maxentry    18
[4] self.len    0
[5] self.addr1  0x00
[6] self.addr2  0x00000000
    self.address        0
[7] next        0x0
[8] prev        0x0
xtree: Hit enter to see entries, [h]ex dump, [u]p, [m]odify, [n]ext, [p]rev, or e[x]it:
xad[2]
    offset     0x0000000000    (0)
    address    0x000000003f    (63)
    len        0x000001        (1)
[1] flag        0x00
[2] off1        0x00
[3] off2        0x00000000
[4] len         0x000001
[5] addr1       0x00
[6] addr2       0x0000003f

 
- As per the above, the address "0x000000003f" (63 in decimal)is
the logical address while reading from the filesystem LV (i.e not
related to the physical address if reading from the disk).
 
- If we continued after the tree information, and ran the "h"
subcommand for hexdump information against this i-node, we’ll
see a one-line description includes offset, length and the logical
Address itself as below:

xtree: Hit enter to see entries, [h]ex dump, [u]p, [m]odify, [n]ext, [p]rev, or e[x]it: h
Index:   Flag Offset        Len       Address
    2:  0x00  0x0000000000  0x000001 0x000000003f


- I can continue by running "s" subcommand to select the index.
For example, in our case the file in index 2:
xtree: Press enter for next, [m]odify, [u]p, [s]elect, or e[x]it > s
Please enter an index from 2 to 17 > 2
xad[2]
    offset     0x0000000000    (0)
    address    0x000000003f    (63)
    len        0x000001        (1)
[1] flag        0x00
[2] off1        0x00
[3] off2        0x00000000
[4] len         0x000001
[5] addr1       0x00
[6] addr2       0x0000003f
xtree: Press [m]odify, [u]p, [s]elect, or e[x]it >
 

Anyway, the logical address is 63 in decimal, and the block size
is 4096 bytes (very obvious in the very beginning of setp_5).
 
6. So, what we are doing now is a dd from this information, knowing that the filesystem is unmounted:
 
# dd if=/dev/fslv00 of=/tmp/file1.logical bs=4096 count=1 skip=63
1+0 records in
1+0 records out
 
# cat /tmp/file1.logical
hello,
This is my test fileeeeeee
 

7. Based on the above, we were able to get the file while the
Filesystem is unmounted.
 
 
8. The other interesting scenario is that we might have the VG
Varied off or reduced for some reason:
 
# cd ; umount /testfs
# reducevg -df testvg hdisk1
rmlv: Logical volume loglv00 is removed.
rmlv: Logical volume fslv00 is removed.
ldeletepv: Volume Group deleted since it contains no physical volumes.
 
# lspv
hdisk0    000f0a9a08aeb344       rootvg          active
hdisk1    000f0a9a0cabbd2e       None
# readvgda hdisk1
Invalid lvm_rec!
00000E00   00000000 00000000 00000000 00000000  |................|
00000E10   00000000 00000000 00000000 00000000  |................|
00000E20   00000000 00000000 00000000 00000000  |................|
00000E30   00000000 00000000 00000000 00000000  |................|
00000E40   00000000 00000000 00000000 00000000  |................|
00000E50   00000000 00000000 00000000 00000000  |................|
00000E60   00000000 00000000 00000000 00000000  |................|
00000E70   00000000 00000000 00000000 00000000  |................|
00000E80   00000000 00000000 00000000 00000000  |................|
00000E90   00000000 00000000 00000000 00000000  |................|
00000EA0   00000000 00000000 00000000 00000000  |................|
00000EB0   00000000 00000000 00000000 00000000  |................|
00000EC0   00000000 00000000 00000000 00000000  |................|
00000ED0   00000000 00000000 00000000 00000000  |................|
00000EE0   00000000 00000000 00000000 00000000  |................|
00000EF0   00000000 00000000 00000000 00000000  |................|
read_lvm_rec had an error, exiting

 
9. Now VGDA is empty at the moment, and it is impossible to import
the VG back:
 
# importvg -y testvg hdisk1
0516-066 lqueryvg: Physical volume is not a volume group member.
Check the physical volume name specified.
0516-066 lqueryvg: Physical volume is not a volume group member.
Check the physical volume name specified.
0516-1140 importvg: Unable to read the volume group descriptor area on specified physical volume.
 

10. But since we have the fileplace output, we can get our files:
 
# fileplace -pv /testfs/file1
File: /testfs/file1  Size: 34 bytes  Vol: /dev/fslv00
Blk Size: 4096  Frag Size: 4096  Nfrags: 1
Inode: 3  Mode: -rw-r--r--  Owner: root Group: system
Physical Addresses (mirror copy 1)              Logical Extent
----------------------------------------------------------------
00528991       hdisk1 1 frags 4096 Bytes,  100.0%    00000063

 
11. Now we won’t use the logical address, because no logical
volume (i.e no filesystem) is present. However, we’ll read
directly from hdisk as the input, using the physical address:
 
# dd if=/dev/hdisk1 of=/tmp/file1.physical bs=4096 count=1 skip=00528991
1+0 records in
1+0 records out
 
# cat /tmp/file1.physical
hello,
This is my test fileeeeeee
 

I am sure, it is not the best - or even a hard task to back up or
Grab the addresses information, as getting the file(s) back from the addresses is not the best or not even
guaranteed.
 
12. In the last scenario I have done; There's a problem in /opt filesystem, which is partially mirrored, and no mksysb backup.
I had to follow the lreducelv to remove the bad copy, but it didn’t work and the issue now is in the mirroring—neither in the VG nor in the exact filesystem blocks.
 
13. List the LV mapping (Copy 2 is the good one):
# lslv -m hd10opt
hd10opt:/opt
LP    PP1 PV1               PP2  PV2               PP3  PV3
0001  0110 hdisk731          0149 hdisk731
0002  0109 hdisk731          0150 hdisk731
0003  0149 hdisk730          0151 hdisk731
0004  0188 hdisk731          0152 hdisk731
0005  0189 hdisk731          0153 hdisk731
0006  0190 hdisk731          0154 hdisk731
0007  0153 hdisk730          0155 hdisk731
0008  0154 hdisk730          0156 hdisk731
0009  0191 hdisk731          0157 hdisk731
0010  0192 hdisk731          0158 hdisk731
0011  0193 hdisk731          0159 hdisk731

 
- We can use the command(s):
# lslv -m hd10opt | awk '{print $2,$3}' | sort -n
# lslv -m hd10opt | awk '{print $2,$3}' | sort -n |tail +3
# lslv -m hd10opt | awk '{print $2,$3}' | sort -n |tail +3 | wc -l
 

14. I created the map file, backed filesystems file:
 
# echo "hdisk731:0149-0159" > opt.map
# cp /etc/filesystems /etc/filesystems.orig

 
15. Deleted /opt (It can be unmounted and deleted while the system is
running. This is not applicable upon /, /usr, /var filesystems)
# umount -f /opt
# rmfs -r /opt

 
16. I created a new logical volume with the same name on the same PP's based on map file created in step 13
 
# mklv -y hd10opt -t jfs2 -m opt.map rootvg 11
# mkdir /opt


17. Next, I added the following lines into /etc/filesystems:
 
/opt:
        dev             = /dev/hd10opt
        vfs             = jfs2
        log             = /dev/hd8
        mount           = true
        check           = true
        vol             = /opt
        free            = false

 
18. Sync the ODM, i-node table:
# lsfs
# synclvodm -Pv rootvg
# sync ; sync ; sync
# fsck -y /opt

 
I was able to mount it normally and see all the old data!
 
The instructions above aren’t a direct solution for the
missing data, but they identify and prove that the data
Isn’t deleted—just the metadata.
 
The metadata is our view and links to the actual data on
different sides. In the Volume Groups, we have the ODM and VGDA information. If the ODM is corrupted against specific VG, it can be recreated by exportvg/importvg, using the VGDA information.
Also, the VGDA information can be edited/cloned and the stale bits can
be zeroed by hexdump tool.
 
In filesystems, we have primary and secondary blocks.
 
Even in Windows FAT/NTFS, if we did a format against a partition,
we can get some tools to recover the data.
The normal format is always removing the metadata tables, which is
the thing we use to see there is an actual data.
 
** Just for testing, I have written a script to collect fileplace
Information for any specific volume group:
 
for i in `lsvgfs $1`
do
echo "=================================================="
echo COLLECTING FILEPLACE INFORMATION FOR "$i" FILESYSTEM
echo "=================================================="
echo ""
find $i -xdev -type f ! -name '.*' -exec fileplace -pv {} \;
echo "************************************************************************"
done >$1.fileplace


It won’t collect the hidden files, but it is up to you anyway to
Edit the script as per your needs. Run the script this way:

# ./script_name VGNAME - It will generate at the end a file named
VGNAME.fileplace – this “could” be helpful if you missed some files.



Like what you just read? To receive technical tips and articles directly in your inbox twice per month, sign up for the EXTRA e-newsletter here.


comments powered by Disqus

Advertisement

Advertisement

2018 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

AIX > ADMINISTRATOR > SYSTEMS MANAGEMENT

How to Download Fixes

ADMINSTRATOR > SYSTEMS MANAGEMENT

Understand your options for 12X PCIe I/O drawers

clmgr: A Technical Reference

PowerHA SystemMirror 7.1 introduces a robust CLI utility

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
IBMi News Sign Up Today! Past News Letters