In compile_and_load C-code is generated twice
When I was reviewing my old code I saw that I placed a TODO here.
generate_c
is just called to generate the hash and then again for the real code (if not loaded from shared object on disk).
Maybe the ast can be hashed directly instead (danger point: generate_c may have changed since last generation).
I remember that I directly hashed the assignments for generation of torch/tensorflow code.
This had the disadvantage that I had to deactivate caching when developing/changing code affecting generate_c
.
def compile_and_load(ast):
cache_config = get_cache_config()
# TODO: inefficient to generate_c just for hash? could reuse it
code_hash_str = "mod_" + hashlib.sha256(generate_c(ast).encode()).hexdigest()
code = ExtensionModuleCode(module_name=code_hash_str)
code.add_function(ast, ast.function_name)
if cache_config['object_cache'] is False:
with TemporaryDirectory() as base_dir:
lib_file = compile_module(code, code_hash_str, base_dir)
result = load_kernel_from_file(code_hash_str, ast.function_name, lib_file)
else:
lib_file = compile_module(code, code_hash_str, base_dir=cache_config['object_cache'])
result = load_kernel_from_file(code_hash_str, ast.function_name, lib_file)
rtn = KernelWrapper(result, ast.get_parameters(), ast)
rtn.code = code.code
return rtn