.TDF Terrain format

[Irup]

I could not find any documentation for it, so I took Xir's suggestion and I've mapped out the basics of the format, but most of it is still a mystery to me. Here's a testing script with my findings for anyone curious, or wanting to do more research. If anyone has any findings that could go into an editor, please post them.

 

Overview:

The terrain is divided into 1024 (32x32) tiles.

There are 4 levels of detail, and each tile's surface consists of a grid of 17x17, 9x9, 5x5 and 3x3 vertices, with each grid popping in and out of view depending on how far you are from it. They can be edited separately.

Each vertex consists of only a height, and 4 bytes, which determine things like lighting and texture blending.

from struct import unpack

infilepath =\
r'C:\Program Files (x86)\LEGO Media\LEGO Racers 2\game data\EDITOR GEN\TERRAIN\SANDY ISLAND\TERRDATA.TDF backup'
outfilepath =\
r'C:\Program Files (x86)\LEGO Media\LEGO Racers 2\game data\EDITOR GEN\TERRAIN\SANDY ISLAND\TERRDATA.TDF'


HEADER_BYTESPER = 0x4
HEADER_OFFSET   = 0x0
HEADER_SIZE     = 0x20

# 0 1 2 3  4  5  6  7
# ffffffff bb bb bb bb
# |        |  |  |  |
# height   |  |  |  texture blending?
#          |  |  light level?????
#          |  flags
#          unknown?????????????
#flags:
# 0x01 0b00000001 = pass-through (all LOD meshes have this checked)
# 0x02 0b00000010
# 0x04 0b00000100
# 0x08 0b00001000
# 0x10 0b00010000
# 0x20 0b00100000
# 0x40 0b01000000
# 0x80 0b10000000
VERTEX_LODS = 4
VERTEX_BYTESPER = 0x8
VERTEX_OFFSET   = (0x20, 0x242020, 0x2e4020, 0x316020)
VERTEX_SIZE     = (0x242000, 0xa2000, 0x32000, 0x12000) # [VERTEX_BYTESPER * lod**2 * WORLD_TILE_LINE**2 for lod in TILE_VERTEX_LINE]
VERTEX_LOD_BYTESPER = [x // 32**2 for x in VERTEX_SIZE]
# unknown
#UNKDA0_BYTESPER = 0xF8     # UNKDA0_SIZE / WORLD_TILE_TOTAL
UNKDA0_OFFSET   = 0x328020 #
UNKDA0_SIZE     = 0x3E000  #
# unknown
UNKDA1_BYTESPER = 0x120    # UNKDA1_SIZE / WORLD_TILE_TOTAL
UNKDA1_OFFSET   = 0x366020 #
UNKDA1_SIZE     = 0x48000  #
# all integers, offsets into tiles from UNKDA1_OFFSET? the numbers are divisible by UNKDA1_BYTESPER
UNKDA2_BYTESPER = 0x4      # UNKDA2_SIZE / WORLD_TILE_TOTAL 
UNKDA2_OFFSET   = 0x3AE020 #
UNKDA2_SIZE     = 0x1000   #
# all floats
UNKDA3_BYTESPER = 0xC8     # UNKDA3_SIZE / WORLD_TILE_TOTAL
UNKDA3_OFFSET   = 0x3AF020 #
UNKDA3_SIZE     = 0x32000  #

TILE_VERTEX_LINE    = (17, 9, 5, 3)
TILE_LENGTH         = [x**2 * VERTEX_BYTESPER    for x in TILE_VERTEX_LINE]
WORLD_TILE_LINE     = 32
WORLD_VERTEX_TOTAL  = [x**2 * WORLD_TILE_LINE**2 for x in TILE_VERTEX_LINE]

DISTANCE = 0.2

def blender_import(): #imports 300000 vertices, but doesn't work too well
	import bpy, bmesh
	f = open(infilepath, 'rb')
	T = TILE_VERTEX_LINE[0]
	terrain_bmesh = bmesh.new()
	for v in range(WORLD_VERTEX_TOTAL[0]):
		terrain_bmesh.verts.new(( #alien code, not safe for human brains
			DISTANCE * ((v % T) + T * ((v // (T**2)) % WORLD_TILE_LINE)),
			DISTANCE * (((v // T) % T) + T * (v // (T**2 * WORLD_TILE_LINE))),
			unpack('f', f.read(4))[0],
		))
		f.read(4)
	terrain_mesh = bpy.data.meshes.new('terraintest mesh')
	terrain_bmesh.to_mesh(terrain_mesh)
	terrain_bmesh.free()
	terrain_object = bpy.data.objects.new('terraintest obj', terrain_mesh)
	bpy.context.scene.objects.link(terrain_object)

def action():
	def showheader():
		f = open(infilepath, 'rb')
		f.seek(HEADER_OFFSET)
		signature = f.read(4).decode('ansi')
		print('Signature:',signature)
		header = [f.read(4) for num in range((HEADER_SIZE-4) // 4)]
		labels = (
			'Unknown 0 ',
			'Unknown 1 ',
			'Unknown 2 ',
			'Add height',
			'Unknown 3 ',
			'Unknown 4 ',
			'Unknown 5 ',
		)
		for label, number in zip(labels, header):
			print('%s: %-10i %-11i %-8x' % (
					label,
					unpack('I', number)[0],
					unpack('i', number)[0],
					unpack('I', number)[0]
				),
				unpack('f', number)[0]
			)
		f.close()
	def findpassthroughvert():
		f = open(infilepath, 'rb')
		o = open(outfilepath, 'wb')
		f.seek(HEADER_OFFSET)
		o.write(f.read(HEADER_SIZE))
		for lod in range(VERTEX_LODS):
			for tile in range(32**2):
				passthroughvertex = False
				for vertex in range(TILE_VERTEX_LINE[lod]**2):
						co = f.read(4)
						unk, flags, light, texblend = f.read(4)
						if not lod and flags & 1: passthroughvertex = (f.tell()-8, flags)
						o.write(co + bytes((0, 0 if passthroughvertex else flags, light, texblend)))
				if passthroughvertex: print('Found pass-through vertex in tile %i, flags: %2x offset: %x' % (tile, passthroughvertex[1], passthroughvertex[0]))
		o.write(f.read())
		f.close()
		o.close()
	def clearflagseveryfifthtile():
		f = open(infilepath, 'rb')
		o = open(outfilepath, 'wb')
		f.seek(HEADER_OFFSET)
		o.write(f.read(HEADER_SIZE))
		for lod in range(VERTEX_LODS):
			for tile in range(32**2):
				if not tile % 5:
					for vertex in range(TILE_VERTEX_LINE[lod]**2):
						co = f.read(4)
						unk, flags, light, texblend = f.read(4)
						o.write(co + bytes((unk, 0, light, texblend)))
				else:
					o.write(f.read(TILE_LENGTH[lod]))
		f.close()
		o.close()
	def makeunkda1homogenous(every = None, copytile = 32*4+16): #can make tiles that crash the game if crossed, tiles that flicker textures, the results are not very predictable
		f = open(infilepath, 'rb')
		o = open(outfilepath, 'wb')
		f.seek(HEADER_OFFSET)
		o.write(f.read(HEADER_SIZE + sum(VERTEX_SIZE) + UNKDA0_SIZE))
		
		f.seek(UNKDA1_OFFSET + UNKDA1_BYTESPER*copytile)
		tilesample = f.read(UNKDA1_BYTESPER)
		
		for tile in range(32**2):
			if every != None and not tile % every:
				o.write(tilesample)
			else:
				f.seek(UNKDA1_OFFSET + UNKDA1_BYTESPER*tile)
				o.write(f.read(UNKDA1_BYTESPER))
			
		f.seek(UNKDA2_OFFSET)
		o.write(f.read())
		f.close()
		o.close()
	makeunkda1homogenous(every = 5)
	input('Finished')
	
action()

 

Here I set all the vertex heights in every fifth tile to zero:

 

Here's the very broken, but almost recognizable blender import:

 

This weird thing happened by making every tile info structure (presumably, UNKDA1) homogenous to tile info 144. I've updated the script to be nicer and have more functions. All the invisible ground was solid.

[Fluffy Cupcake]

Ah welp, you've got more on it than I have from my several times poking at it, bravo! x)

 

  Quote

Here I set all the vertex heights in every fifth tile to zero:

Expand  

Ooh - if it ain't too much, think you can quickly whip up a flat map? I tried setting all bytes to 0 once, but I got these spiked up walls on the edges of the tiles for some reason.

 

You know, I noticed something, sometimes the terrain can have cutouts, I wonder how that is done? If you find an answer, let me know.

[Irup]

I don't know where the tile borders are stored, but they're not in the vertex chunk, known as VERTEX_OFFSET in the script. Perhaps they're in UNKDA1 or UNKDA3, or even the TDF's accompanying OCCLUSION DATA file, which I think determines what chunks are rendered depending on where you are, as a rendering optimization. That would need to be explored as well to make a future flatgrass map less annoying.

 

The hole, like your screenshot shows, is fillable by setting the vertex flags to 0:

[Fluffy Cupcake]

Thanks, but I was actually wondering the opposite. What do the bytes/value look like that make it non-filled?

[Bobberto1995]

After not finding the level files on models-resource, I decided to do a little digging into the TDF format and found this thread.  Thanks to lrup for the script and research! I have created a Github repository that contains my script that will extract the components of a TERRDATA.TDF file into usable file formats.

 

Example images (Sandy Bay aka SANDY ISLAND at mipmap level 0):
https://imgur.com/a/iyCEcpp

 

[lol username]

From Discord the other day:

 

[10:50 AM] ruta: 

Hi all guys, I'm the guy that some time ago created a tool able to import LR2 .TDF models into Blender. For who who doesn't know, TDF is the format LR2 uses for terrains. At first I was hesitant to share it since I wanted to work on LR2 terrains on my own during this summer, but since I've noticed I don't have too much time I've decided to give you the link and all the work I've done so far: https://github.com/lorenzorutayisire/lr2-blender-loader