目录
主要参考文章:linux之kasan原理及解析-CSDN博客
之前使用slub debug定位重复释放,内存越界等问题时比较麻烦。无法对异常行为进行实时捕捉。看网上说kasan能做到这一点。现在准备看一下kasan是如何能够做到这一点的。
Kernel Address SANitizer(KASAN)是一个动态检测内存错误的工具。能够检测到释放后使用、堆、栈、全局变量越界访问等问题(slub debug就无法做到这么全面。它只能检查到从slab分配器里面申请的内存)。
那kasan是如何能够检测到上述情况的呢?
其原理就是利用额外的内存,去标记内存的可用状态。这部分用于记录内存可用状态的内存被称为shadow memory。shadow memory和正常内存比例是1:8(可以看到kansan是比较消耗内存,影响代码执行效率的。slub debug也是会增加内存消耗。对于小内存设备,这些内存检测工具可能都无法使用)。这部分内存里面被记录了一些特殊的值。当每次对内存进行读写时,就会去检查这个地址对应的shadow memory里面的状态(这个读写检查的动作据说是编译器直接进行插入的)。这样就能检测到此次读写是否有问题。
shadow memory里面填写规则:连续字节的内存,需要用1字节shadow memory标记。
1、如果这8字节内存都可以访问,那么1字节的shadow memory填写为0;
2、如果连续N(1<= N<= 7)字节可以访问,则shadow memory值为N;
3、如中只能果这8字节内存都无法访问,则shadow memory为负数(负值具体是多少呢);
gcc4.8中引入了一个新的内存错误检测工具:AddressSanitizser,使用-fsanitize=address。这个工具是用于在运行时检查c/c++内存错误。它核心机制就是在所有内存读写之前,插入一个判断权限的钩子函数。
AddressSanitizer&ThreadSanitizer原理与应用 - 知乎
kasan就是利用了这个特性。在内核中实现了这种功能
代码样例:局部变量越界访问
#include <stdio.h> #include <stdlib.h> int main(int argc, char* argv) { int a[10]= {0}; a[11] = 15; return 0; }
gcc addrsantizer.c ?-fsanitize=address -o addr
==38542==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffcea1986fc at pc 0x555f83023ae3 bp 0x7ffcea198690 sp 0x7ffcea198680
WRITE of size 4 at 0x7ffcea1986fc thread T0
? ? #0 0x555f83023ae2 in main (/Test/addr+0xae2)
? ? #1 0x7f4866f8fc86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)
? ? #2 0x555f83023899 in _start /Test/addr+0x899)Address 0x7ffcea1986fc is located in stack of thread T0 at offset 76 in frame
? ? #0 0x555f83023989 in main /Test/addr+0x989)? This frame has 1 object(s):
? ? [32, 72) 'a' <== Memory access at offset 76 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
? ? ? (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow Test/addr+0xae2) in main
Shadow bytes around the buggy address:
? 0x10001d42b080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b0a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b0b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b0c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10001d42b0d0: 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00 00[f2]
? 0x10001d42b0e0: f2 f2 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b0f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
? 0x10001d42b120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
? Addressable: ? ? ? ? ? 00
? Partially addressable: 01 02 03 04 05 06 07?
? Heap left redzone: ? ? ? fa
? Freed heap region: ? ? ? fd
? Stack left redzone: ? ? ?f1
? Stack mid redzone: ? ? ? f2
? Stack right redzone: ? ? f3
? Stack after return: ? ? ?f5
? Stack use after scope: ? f8
? Global redzone: ? ? ? ? ?f9
? Global init order: ? ? ? f6
? Poisoned by user: ? ? ? ?f7
? Container overflow: ? ? ?fc
? Array cookie: ? ? ? ? ? ?ac
? Intra object redzone: ? ?bb
? ASan internal: ? ? ? ? ? fe
? Left alloca redzone: ? ? ca
? Right alloca redzone: ? ?cb
==38542==ABORTING
下图是inux5.4 .1(arm64)开启kasan后部分函数汇编代码的详细情况。可以看到代码里面被插入了__asan_xx这些代码
aarch64-linux-gnu-objdump -d msm_serial.o
__asan_xxx:这些是如何实现的呢??详细代码如下:
./mm/kasan/generic.c?
#define DEFINE_ASAN_LOAD_STORE(size) \
void __asan_load##size(unsigned long addr) \
{ \
check_memory_region_inline(addr, size, false, _RET_IP_);\
} \
EXPORT_SYMBOL(__asan_load##size); \
__alias(__asan_load##size) \
void __asan_load##size##_noabort(unsigned long); \
EXPORT_SYMBOL(__asan_load##size##_noabort); \
void __asan_store##size(unsigned long addr) \
{ \
check_memory_region_inline(addr, size, true, _RET_IP_); \
} \
EXPORT_SYMBOL(__asan_store##size); \
__alias(__asan_store##size) \
void __asan_store##size##_noabort(unsigned long); \
EXPORT_SYMBOL(__asan_store##size##_noabort)
__asan_load8_noabort其实就是__asan_load8的别名
测试样例
#include <stdio.h>
#include <stdlib.h>
void func()
{
printf("\t\tthis is func\n");
}
#define __alias(symbol) __attribute__((__alias__(#symbol)))
__alias(func) void func_alias();
int main(int argc, char* argv)
{
func_alias();
return 0;
}
kasan里面有一块shadow memory。要实现内存的检查,必须要先有这块区域记录内存读写权限(就是上面将的8字节内存是否有效的标记信息)的详细信息,这块区域是如何划分、何时划分的呢?
kasan_init
void __init kasan_init(void)
{
u64 kimg_shadow_start, kimg_shadow_end;
u64 mod_shadow_start, mod_shadow_end;
struct memblock_region *reg;
int i;
kimg_shadow_start = (u64)kasan_mem_to_shadow(_text) & PAGE_MASK;
kimg_shadow_end = PAGE_ALIGN((u64)kasan_mem_to_shadow(_end));
mod_shadow_start = (u64)kasan_mem_to_shadow((void *)MODULES_VADDR);
mod_shadow_end = (u64)kasan_mem_to_shadow((void *)MODULES_END);
/*
* We are going to perform proper setup of shadow memory.
* At first we should unmap early shadow (clear_pgds() call below).
* However, instrumented code couldn't execute without shadow memory.
* tmp_pg_dir used to keep early shadow mapped until full shadow
* setup will be finished.
*/
memcpy(tmp_pg_dir, swapper_pg_dir, sizeof(tmp_pg_dir));
dsb(ishst);
cpu_replace_ttbr1(lm_alias(tmp_pg_dir));
clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END);
printk("_text 0x%llx, _end 0x%llx, kimg_shadow_start 0x%llx, kimg_shadow_end 0x%llx\n", \
_text, _end, kimg_shadow_start, kimg_shadow_end);
printk("MODULES_VADDR 0x%llx, MODULES_END 0x%llx, mod_shadow_start 0x%llx, mod_shadow_end 0x%llx\n", \
MODULES_VADDR, MODULES_END, mod_shadow_start, mod_shadow_end);
printk("KASAN_SHADOW_SCALE_SHIFT 0x%llx, KASAN_SHADOW_OFFSET 0x%llx,PAGE_END 0x%llx, KASAN_SHADOW_START 0x%llx, KASAN_SHADOW_END 0x%llx\n", \
KASAN_SHADOW_SCALE_SHIFT, KASAN_SHADOW_OFFSET, PAGE_END, KASAN_SHADOW_START, KASAN_SHADOW_END);
kasan_map_populate(kimg_shadow_start, kimg_shadow_end,
early_pfn_to_nid(virt_to_pfn(lm_alias(_text))));
kasan_populate_early_shadow(kasan_mem_to_shadow((void *)PAGE_END),
(void *)mod_shadow_start);
kasan_populate_early_shadow((void *)kimg_shadow_end,
(void *)KASAN_SHADOW_END);
if (kimg_shadow_start > mod_shadow_end)
kasan_populate_early_shadow((void *)mod_shadow_end,
(void *)kimg_shadow_start);
for_each_memblock(memory, reg) {
void *start = (void *)__phys_to_virt(reg->base);
void *end = (void *)__phys_to_virt(reg->base + reg->size);
if (start >= end)
break;
kasan_map_populate((unsigned long)kasan_mem_to_shadow(start),
(unsigned long)kasan_mem_to_shadow(end),
early_pfn_to_nid(virt_to_pfn(start)));
}
/*
* KAsan may reuse the contents of kasan_early_shadow_pte directly,
* so we should make sure that it maps the zero page read-only.
*/
for (i = 0; i < PTRS_PER_PTE; i++)
set_pte(&kasan_early_shadow_pte[i],
pfn_pte(sym_to_pfn(kasan_early_shadow_page),
PAGE_KERNEL_RO));
memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE);
cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
/* At this point kasan is fully initialized. Enable error messages */
init_task.kasan_depth = 0;
pr_info("KernelAddressSanitizer initialized\n");
}
将kasan_init里面用到的几个地址?打印了出来(kasan_map_populate这个函数就是做映射的)
感觉kasan_init就是将下图画的几个区域做映射。但是我并没有看到对mod_shadow_start到mod_shadow_end这个区域做映射呢?这个部分我完全没有看明白。
另外我看网上提到类似于下图的区域,也没有看到在哪里有体现。。。。放弃,后面在研究吧
继续看?void __asan_load##size(unsigned long addr)的实现
static __always_inline bool check_memory_region_inline(unsigned long addr,
size_t size, bool write,
unsigned long ret_ip)
{
if (unlikely(size == 0))
return true;
/* 检查地址对应的shadow 地址是否正确 */
if (unlikely((void *)addr <
kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) {
kasan_report(addr, size, write, ret_ip);
return false;
}
/* 判断权限是否正确 */
if (likely(!memory_is_poisoned(addr, size)))
return true;
/* 打印异常情况 */
kasan_report(addr, size, write, ret_ip);
return false;
}
如何判断内存学些是否存在问题呢?
static __always_inline bool memory_is_poisoned(unsigned long addr, size_t size)
{
if (__builtin_constant_p(size)) {
switch (size) {
case 1:
return memory_is_poisoned_1(addr);
case 2:
case 4:
case 8:
return memory_is_poisoned_2_4_8(addr, size);
case 16:
return memory_is_poisoned_16(addr);
default:
BUILD_BUG();
}
}
return memory_is_poisoned_n(addr, size);
}
memory_is_poisoned_1?
static __always_inline bool memory_is_poisoned_1(unsigned long addr)
{
s8 shadow_value = *(s8 *)kasan_mem_to_shadow((void *)addr);
/* 0表示8字节都可访问,N表示连续N字节可访问 */
if (unlikely(shadow_value)) {
/*
取最后3bit,看此次访问的1字节内存所处的位置
目前我们有连续shadow_value字节可以访问,如果addr>= shadow_value
则表示这个位置他不能访问
*/
s8 last_accessible_byte = addr & KASAN_SHADOW_MASK;
return unlikely(last_accessible_byte >= shadow_value);
}
return false;
}
?
static __always_inline bool memory_is_poisoned_2_4_8(unsigned long addr,
unsigned long size)
{
u8 *shadow_addr = (u8 *)kasan_mem_to_shadow((void *)addr);
/*
* Access crosses 8(shadow size)-byte boundary. Such access maps
* into 2 shadow bytes, so we need to check them both.
*/
/*
大概意思是此时访问的内存,在两字节的shadow memoryl里面.所以两个都需要检测
只要有一个为true,就表示有问题,不可访问
*/
if (unlikely(((addr + size - 1) & KASAN_SHADOW_MASK) < size - 1))
return *shadow_addr || memory_is_poisoned_1(addr + size - 1);
/* 只在一字节的shadow memory里面 */
return memory_is_poisoned_1(addr + size - 1);
}
memory_is_poisoned_2_4_8:示意图
?? ?if (unlikely(((addr + size - 1) & KASAN_SHADOW_MASK) < size - 1))
?? ??? ?return *shadow_addr || memory_is_poisoned_1(addr + size - 1);跨两字节,那就需要两字节都允许访问。所以第一个shadow memory必须为0(N表示连续N字节都可访问。6,7都能访问的话,那就是8字节都能访问了,所以为0)
memory_is_poisoned_1(addr + size - 1)//为什么这里只用检测1字节就行了呢
还是用上图.假设第二个shadow memory的第1字节可访问,那必然0字节也是可以访问的(N表示连续N字节都可访问。)
memory_is_poisoned_16和访问更大的范围就不解析了。?
接下来就是发现问题,打印log
kasan_report
void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip)
{
unsigned long flags = user_access_save();
__kasan_report(addr, size, is_write, ip);
user_access_restore(flags);
}
alloc_pages() -->__alloc_pages_node()-->__alloc_pages() --?>__alloc_pages_nodemask() -->get_page_from_freelist-->?prep_new_page()-->post_alloc_hook() ->kasan_alloc_pages()
我代码里没有定义这个宏#ifdef CONFIG_KASAN_SW_TAGS?
void kasan_alloc_pages(struct page *page, unsigned int order)
{
u8 tag;
unsigned long i;
if (unlikely(PageHighMem(page)))
return;
tag = random_tag();//kasan.h
for (i = 0; i < (1 << order); i++)
page_kasan_tag_set(page + i, tag);
kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order);
}
伙伴系统申请是以page为单位,肯定是8字节对齐的。它不走下面?
void kasan_unpoison_shadow(const void *address, size_t size)
{
u8 tag = get_tag(address);
/*
* Perform shadow offset calculation based on untagged address, as
* some of the callers (e.g. kasan_unpoison_object_data) pass tagged
* addresses to this function.
*/
address = reset_tag(address);//其实就是原封不动返回address
kasan_poison_shadow(address, size, tag);//向内存里面填入tag,其实这里是0
/大小未对齐,需要单独对最后一字节进行处理/
if (size & KASAN_SHADOW_MASK) {
u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
*shadow = tag;
else
*shadow = size & KASAN_SHADOW_MASK;
}
}
这里入参tag是0,因此可以看到从伙伴系统里面申请出来的page,他们对应的shadow memory的值为0(本来也应该为0,因为全部内存都允许被访问)
void kasan_poison_shadow(const void *address, size_t size, u8 value)
{
void *shadow_start, *shadow_end;
/*
* Perform shadow offset calculation based on untagged address, as
* some of the callers (e.g. kasan_poison_object_data) pass tagged
* addresses to this function.
*/
address = reset_tag(address);
shadow_start = kasan_mem_to_shadow(address);
shadow_end = kasan_mem_to_shadow(address + size);
__memset(shadow_start, value, shadow_end - shadow_start);
}
__free_pages-->__free_pages_ok-->free_pages_prepare-->kasan_free_nondeferred_pages-->kasan_free_pages
可以看到在page释放之后,其对应的shadow memroy里面的值会被填为0xff
#define KASAN_FREE_PAGE 0xFF /* page was freed */
void kasan_free_pages(struct page *page, unsigned int order)
{
if (likely(!PageHighMem(page)))
kasan_poison_shadow(page_address(page),
PAGE_SIZE << order,
KASAN_FREE_PAGE);
}
void kasan_poison_shadow(const void *address, size_t size, u8 value)
{
void *shadow_start, *shadow_end;
/*
* Perform shadow offset calculation based on untagged address, as
* some of the callers (e.g. kasan_poison_object_data) pass tagged
* addresses to this function.
*/
address = reset_tag(address);
shadow_start = kasan_mem_to_shadow(address);
shadow_end = kasan_mem_to_shadow(address + size);
__memset(shadow_start, value, shadow_end - shadow_start);
}
所以如果是在访问的时候发现shadow memory是0xff,那就说明是释放后再使用
1、new_slab-->new_slab-->kasan_poison_slab:将得到的整个page对应的shadow memory全部统一初始化为KASAN_KMALLOC_REDZONE=0xFC?(这里应该是包含了obj size和一些用于debug的内存,比如slub debug)。后面研究一下这个new slab和slab池子的关系
void kasan_poison_slab(struct page *page)
{
unsigned long i;
for (i = 0; i < compound_nr(page); i++)
page_kasan_tag_reset(page + i);
kasan_poison_shadow(page_address(page), page_size(page),
KASAN_KMALLOC_REDZONE);
}
2、在实际进行申请的时候,区分了实际使用的区域和开启kasan所增加的redzone
__kmalloc-->__kasan_kmalloc
void *__kmalloc(size_t size, gfp_t flags)
{
.....................
ret = slab_alloc(s, flags, _RET_IP_);
trace_kmalloc(_RET_IP_, ret, size, s->size, flags);
ret = kasan_kmalloc(s, ret, size, flags);
return ret;
}
可以看到申请到obj之后,会?调用__kasan_kmalloc,将实际申请的内存,所对应的shadow memory区域初始化为0,表示全部都可访问。紧跟在后面有个redzone,其shadow memory初始化为0xfc
static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
size_t size, gfp_t flags, bool keep_tag)
{
unsigned long redzone_start;
unsigned long redzone_end;
u8 tag = 0xff;
if (gfpflags_allow_blocking(flags))
quarantine_reduce();
if (unlikely(object == NULL))
return NULL;
/*
kmem_cache->size是对齐之后的大小
kmem_cache->object_size是对象实际大小(应该是kmem_cache_create传入的大小)
感觉就是在size(用户实际申请的区域)和obj_size之间加了一个大小变化的redzone
如果我刚好申请了512的内存,那感觉redzone可能会没有呢??
还是说kmem_cache->size一定会大于kmem_cache->object_size,所以redzone一定存在??
*/
redzone_start = round_up((unsigned long)(object + size),
KASAN_SHADOW_SCALE_SIZE);
redzone_end = round_up((unsigned long)object + cache->object_size,
KASAN_SHADOW_SCALE_SIZE);
if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
tag = assign_tag(cache, object, false, keep_tag);
/* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */
/* 将可用区域填写为0 */
kasan_unpoison_shadow(set_tag(object, tag), size);
/* 将redzone填写为0xFC */
kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
KASAN_KMALLOC_REDZONE);
if (cache->flags & SLAB_KASAN)
set_track(&get_alloc_info(cache, object)->alloc_track, flags);
return set_tag(object, tag);
}
/*
?? ?kmem_cache->size是对齐之后的大小
?? ?kmem_cache->object_size是对象实际大小(应该是kmem_cache_create传入的大小)
?? ?感觉就是在size(用户实际申请的区域)和obj_size之间加了一个大小变化的redzone
?? ?如果我刚好申请了512的内存,那感觉redzone可能会没有呢??
?? ?还是说kmem_cache->size一定会大于kmem_cache->object_size,所以redzone一定存在??
?? ?*/
上面这段话是多虑了。在创建的时候,就已经考虑了开启kasan占用的red zone。
kmem_cache_open-->calculate_sizes-->kasan_cache_create-->optimal_redzone
void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
slab_flags_t *flags)
{
..........................
redzone_size = optimal_redzone(cache->object_size);
redzone_adjust = redzone_size - (*size - cache->object_size);
.........................
*flags |= SLAB_KASAN;
}
static inline unsigned int optimal_redzone(unsigned int object_size)
{
if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
return 0;
return
object_size <= 64 - 16 ? 16 :
object_size <= 128 - 32 ? 32 :
object_size <= 512 - 64 ? 64 :
object_size <= 4096 - 128 ? 128 :
object_size <= (1 << 14) - 256 ? 256 :
object_size <= (1 << 15) - 512 ? 512 :
object_size <= (1 << 16) - 1024 ? 1024 : 2048;
}
?所以在开启kasan时内存布局如下图
假设我们kmalloc(obj size)。我们使用的大小也只有obj size,但是后面会跟一个red zone.这两个区域都有对应shadow memory进行保护
slab释放
kfree-->slab_free-->slab_free_freelist_hook-->slab_free_hook-->__kasan_slab_free?
1、可以看到在释放的时候,会去检查一下shadow memory是不是正确的。
2、将object_size区域的shadow memroy区域设置为KASAN_KMALLOC_FREE(0xFB)
个人感觉重复释放在这检查,访问越界就由编译器插入的读写检查指令,实时进行检查
static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
unsigned long ip, bool quarantine)
{
...................
shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object));
if (shadow_invalid(tag, shadow_byte)) {
kasan_report_invalid_free(tagged_object, ip);
return true;
}
rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE);
kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
unlikely(!(cache->flags & SLAB_KASAN)))
return false;
kasan_set_free_info(cache, object, tag);
quarantine_put(get_free_info(cache, object), cache);
return IS_ENABLED(CONFIG_KASAN_GENERIC);
}
样例
int arr[10] = {0};
static int __init msm_serial_init(void)
{
int ret;
............................
pr_info("xxx msm_serial: driver initialized\n");
arr[10] = 10;
return ret;
}
大概意思就是每个全局变量后面都会额外增加一个red zone。
1、red zone大小计算规则:全局变量实际占用内存总数S(以byte为单位)
redzone = 63 – (S - 1) % 32
数组大小40字节。40+redzone(63-(40-1) %32)=96
0xC0-0x60=96
2、全局变量shadow memroy初始化:_GLOBAL__sub_I_65535_1_##global_variable_name。编译器会对每个变量都创建一个函数。在这个里面进行初始化
详细的参考文章:一文搞懂Linux内核内存管理中的KASAN实现原理 - 知乎
基本原则还是实际使用内存所对应的的shadow memroy用于检查访问权限,redzone用检测越界问题?
static void register_global(struct kasan_global *global)
{
size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);
kasan_unpoison_shadow(global->beg, global->size);
kasan_poison_shadow(global->beg + aligned_size,
global->size_with_redzone - aligned_size,
KASAN_GLOBAL_REDZONE);
}
void __asan_register_globals(struct kasan_global *globals, size_t size)
{
int i;
for (i = 0; i < size; i++)
register_global(&globals[i]);
}
局部变量是在其前面加32字节的redzone,在其后面加大小为63 – (S - 1) % 32的redzone。用于检查左右越界问题