Leaderboard
Popular Content
Showing content with the highest reputation on 07/19/2012 in Posts
-
TUTORIAL: Using a Hex Editor to Analyze Binary Files
McJobless and 2 others reacted to legomoe for a topic
- Segatendo wanted me to post a topic on this subject, so I've taken the liberty of writing a small tutorial on how to analyze binary files using a program called a 'hex editor'. Analyzing binary files used by a video game (like LEGO Island 2) can be helpful for writing modding tools, because once you understand how the different binary file formats used by a game are structured, you (or someone you share the information with) can write a program to convert that type of file to a more usable format (for example: converting the binary LEGO Island 2 '.msh' format into the text-based and widely used '.obj' format). - To start, you're going to need a hex editor. I personally use Hexplorer for windows. but you can use any hex editor you want. (Note: if you use Hexplorer, on some computers, the default font used by the program is too small to read. To fix this, launch Hexplorer, then in the menu bar, under "View" select "Options...". This should open a windows that lets you pick the font used by the editor. You can also change the editor's text color scheme from this window as well.) - You can analyze any binary file you want, but for this tutorial I'll be using a file from LEGO Island 2. The 'Pizza.col' file from the 'col' directory inside the z02LGI.bod/z02LGI.bob archive. If you want to get this file for yourself, you'll need to download my LI2Explorer tool (http://www.mediafire...34oy5quwy5qtsc4) and use it to open the z02LGI.bod/z02LGI.bob archive located in the '_data' directory of the LEGO Island 2 folder (The path should be "C:\Program Files\LEGO Media\LEGO Island 2\_data"). - Once you open the z02LGI.bod/z02LGI.bob archive with LI2Explore, in the LI2Explorer's menu bar under 'Edit' select 'Export' and then choose a location on your hard drive (I like to use my Desktop for things like this). After a short pause, this should create a folder named 'z02LGI' at the location you specified. Inside the new 'z02LGI' folder should be another folder called 'col', the 'Pizza.col' file should be inside this directory. - Open the 'Pizza.col' file with Hexplorer and you should see something like this: On the left hand side is the hexadecimal representation of the contents of the binary file, and on the right is the exact same data, only represented as ASCII characters. You're going to need both of these representations to analyze the file. - Now, the first thing you need to keep in mind when trying to analyze a binary file is what kind of data is the binary file going to contain. Is it an image file of some sort? Is it an audio file? Does it hold data for a 3D model? or maybe its an archive used to hold other files and folders? Once you know the type of data being represented in the binary file, you can start to figure out exactly how it's organized, and how to write a program to extract the data from the file. - For our example, it can be guessed that the '.col' files in the 'col' directory contain collision data for the game's 3D models, because col is probably short for collision, and since most all the '.msh' files in the meshes directory that would need collision data have a corresponding '.col' file in the 'col' directory. - Now, before you go any further, it's important to understand about basic data types and how they are represented in a binary file. While a binary file may be a very long string of 1s and 0s (bits), for practical purposes they are treated as a string of 8-bit bytes. Again, notice on the left side of the Hexplorer window you can see the individual bytes that make up the file 43 42 42 58 00 7E 42 BB ... etc, represented in hexadecimal (base 16) notation, and on the right, the exact same bytes are represented as ASCII characters. These bytes, alone or in groups, can be use to represent all sorts or data from characters to numbers to strings of text. Here's a basic overview of the different common data types stored in binary files: Name:-------------------Number of Bytes:-----Values: (Can signed (- and +) or unsigned (just +)) Character---------------1--------------------0 to 255 or -127 to 127 Short or Word-----------2--------------------0 to 65535 or -32767 to 32767 Integer-----------------4--------------------0 to 4294967295 or -2147483647 to 2147483647 Floating Point----------4--------------------(Varies based on representation) - Note: When you put bytes together to form larger data types, you need to think about what order the bytes are stored in (Endianness). There are 2 different ways to order bytes: One is known as Little-endian, where the lest significant byte is stored first. the other is known as Big-endian, where the most significant byte is stored first. For example, take the bytes 00 and 10. If these two bytes are read as being a Little-endian short, they would be equal to hexadecimal 1000 (decimal 4096), because the 00 byte is stored first, it is the least significant byte, therefor it goes behind the 10 byte when you read the short. Now, if we interpret these same 2 bytes as being a Big-endian short, they would be equal to hexadecimal 0010 (decimal 16). This is because the 00 byte, being stored first, is the most significant byte, therefore it goes before the 10 byte when you read the short. - Just so you know, most windows computers use Little-endian, while Macs use Big-endian. I also know that all the binary files used by LEGO Island 2 are meant to be read at Little-endian. - Part of analyzing a binary file is being able to figure out what kind of data types are being represented by the bytes. For our example, lets take a look at the first four bytes: 43 42 42 58, These bytes could represent the decimal values 67, 66, 66, and 88, but they could also represent the two short values 16963 and 22594, or the single 4 byte integer value 1480737347. - Now, if you look to the right hand side of the Hexplorer window, you see that these four bytes 43 42 42 58 form the ASCII characters C, B, B, and X. All of these are printable ASCII characters, which means that these first four bytes may be representing a text string, rather than a number. I know from experience that binary files often begin with a 4 character string that identifies what kind of file it is, so it's a pretty safe bet to assume that the first four characters are mean to be read as "CBBX". - On a side note, not all binary files begin with 4 character file identifiers, and many binary files are often divided into sections beginning with 4 character identification strings. So the fact that the 'Pizza.col' file begins with "CBBX" does not necessarily mean that the entire file is a "CBBX" file, it may simply mean the the file does not have a 4 byte string at the beginning, and that "CBBX" is just the first section to appear in the file. - The reason that binary files and sections within those files have these sorts of 4 byte ASCII strings is because when a program reads a binary file, it needs some way to know for sure what kind of file its reading, and where sections may appear within the file. Remember, the 4 byte ASCII strings that appear at the beginning of the file or at the start of sections is often followed by information that can be use to analyze the file or section, such as the number of sections within the file, or how large the particular section of the file may be. - Now if you examine the next 28 bytes of the file, you'll see that if you treat them as 4 byte integers, they all represent arbitrary, excessively large numbers, and if you look to the right side of the Hexplorer window, these 28 bytes don't form a printable ASCII string. So what do these bytes represent? I personally don't know, but I do know that if you just can't seem to figure out what certain bytes represent, it may be better to move on to a different part of the the file until you have an insight into what said bytes represent. - Take a look at the bytes just beyond the first 32 bytes. The first four are 43 4F 4C and 50, and the next four are 40 00 00 and 00. Since the first four form the ASCII string "COLP", we can assume that this is the start of the "COLP" section. Now, since this is the start of a section, often the next few bytes will give us a clue as to how big this section is (don't forget that a computer program needs to read this file, and it can't magically know how big a section of a file will be). - The four bytes after 43 4F 4C and 50 may hold the key to the length of the "COLP" section. If you read 40 00 00 and 00 as a 4 byte integer, you get decimal 64. Now, this probaly doesn't mean that the "COLP" section is 64 bytes long, since if we look 64 bytes past 40 00 00 and 00, you can't see any 4 byte ASCII string to mark the beginning of a new section. However, the 64 may mean that the "COLP" section is made up of 64 sub-sections. - This is where its important to look for patterns in binary files, since if you look at the next 88 bytes, you will see many elements repeat themselves. The next 88 bytes begin with 23 00 00 00 00 00 00 00, and end with four sets of CC CC CC CC. Now if we look at the 88 bytes after this, we see the exact same thing, and if we look at the 88 bytes after that, the pattern repeats. - Now we need to do a little math. If the "COLP" section is supposedly 64 sub-sections long, and each sub-section is 88 bytes long, whats 64 times 88? The answer is 5632. Now in Hexplorer if you click and hold at the beginning of the first sub-section (The bytes 23 00 00 00 right after the "COLP" ASCII string and the bytes '40 00 00 00' that tell you the length of the section) and then scroll down with the mouse wheel till you have exactly 5632 bytes selected (it should tell you how many bytes you have selected at the bottom of the window). The next four bytes past the end of the 5632 selected bytes should be 47 52 44 and 50, which form the ASCII string "GRDP" and signify the start of a new section. - Congradulations! You now know that the second section of the 'Pizza.col' file begins with the 4 byte ASCII string "COLP", and the next four bytes represent a 4 byte integer that tells you the length of the section in 88 byte sub-sections. This is as far as I'm going to go in this tutorial, but if you feel like continuing on your own, you could try and figure out what sort of data is stored in each of the 88 byte sub-sections. Remember that 3D collision data is commonly stored as a simplified 3D mesh, so you might try looking for vertex location information in each of the sub-sections, but I can't guaranty that that's what you'll find. - These are the basics of how you analyze a binary file with a hex editor. It takes a lot of time, and you might not always be able to figure out every part of a file, but you can still learn plenty of useful information about how the file, and other files like it, may be structured. - If you know Assembly language, an easier way to analyze a binary file is to disassemble the program that loads said binary file and examine how the program goes about loading the file. - I hope you found this tutorial helpful. If you have any questions or comments, please feel free to post them in the topic here. Legomoe3 points -
Greetings.
Storm and one other reacted to Cirevam for a topic
That's right. fun is not allowed here, nor is free thinking.2 points -
RRU Quotes 2008-2013
s0d3rb3rg reacted to TheEPICtrainrider for a topic
Post RRU quotes. With the -in regards to the Creation of this topic. -In regards to my 'stupid' comment.1 point -
Rock Raiders United Incorporated Research - Help Wanted!
Jimbob reacted to McJobless for a topic
Rock Raiders United Incorporated is a private company, founded, funded and run by Mayor McCheese Cyrem. It was created on the 31st of July in 2008, and was run by Cyrem's father, gmjab. gmjab, being only a poor man, was able to find some free office space in the city of Brisbane, and used that to expand on his finding of the LEGO creation, LEGO Rock Raiders. gmjab died a little over a year later, but his son, Cyrem, created a new building from scratch with the profit his father had made him. Almost 3 years of progress later, the business has expand to millions of floors, becoming it's own entirely new state in Australia (called "Mexicongo"). The business has 1220 recorded employees, almost 65,000 posts produced and has successfully blocked more than 10,000 attacks. RRU's main source of income is from producing and selling forum posts. The building spans over 32,000,000 floors, and every floor contains at least one toilet. All toilets are in-sync, so if one breaks, they all break. Watercoolers are next to every single toilet block. Important floors are documented below; undocumented floors have either been abandoned or remain undiscovered caverns, containing high traces of ore, crystal and Brickonium. Several members have bought out entire floors, but all members sleep in private rooms in the barracks. 1 floor is square in shape approximate to more than the size of 20 A380 wings tip to tip. Every floor has a pants collection bin which automatically empties into the pants chute. All members are required to submit their pants to this bin upon the entering of a level. Staff are exempt. Level -1.340780793×10¹âµâ´ - Addictgamer's secret lab. It's the basement of RRU. The elevator can't go any lower due to it being at the lowest end of the 512 bit integer, and 512 bits is the largest integer the elevator supports. This level cannot be accessed under normal conditions. You must emag the elevator to unlock 512 bit mode. Some say the reason people (spammers, trolls, bots, etc) disappear from the prison is because addict kidnaps them at night, takes them down to his secret lab, and performs illegal experiments on them. They say he used some of these experiments to create force fields that keep spambots out of the building. Level -32,000,001 - Safe Room, in case of emergency all units should teleport here. Level -32,000,000 - Night Club Archon, IceHusky2's second DJ estate. Level -404 - Everything. Level -322: The holding area and private quarters for the Ultimate Life Form also known as Shadow322. Level -29 - Stargate Command. Level -7 - ftgsarge's level for his shenanigans and stuff. Level -5 - Lair's giant volcano robot living in a lava pool. Level -4 - Watering Hole (only area with natural water). Level -3 - Death Course - Filled with barbed wire, mine fields, snipers and radioactive waste. Aim is to open up a safe Level -2 - Gas Room - Funny gas that makes you invincible. Level -1 - Basement, contains several support stations. Most link strangely to Level 16, and the air tanks seem easy to replace. Level 0 (Ground) - Main entry point to the building. Staff here handle registrations, and employees come down here to handle technical support at the HelpDesk Kiosk. Level 1 - Newbie Floor. Members with less than 50-75 posts are only allowed here and Level 0, for safety reasons. Level 2 - Main Floor. Most members work on this floor, doing file research and other things. Level 3 - Office Amenities, including Extreme's DJ Office (radio), the Blogosphere, a secret cave containing the Knowledge Bricks (wikis) which can only be accessed by worthy members and the Search Bar. It is rumoured that seamonster spends most of his time at this bar, getting drunk and searching things, but nobody can determine what. Level 4 - VIP Section. Level 5 - Moderation Office. Level 6 - Prison and Banhammer Holding Cell. Currently suffering radiation leak, but is under repairs. Level 7 - Extreme's Floor. Level 8 - Was the old Shoutbox, but is currently suffering an unfixable radiation leak. Level 9 - Pascalgames Room. Level 10 - Fanfiction Library. A recent fire destroyed the remaining copies of Matter Shift. Level 11 - Devil's Toybox. Accessible to only certain members, contains dangerous secrets. Level 12 - Segatendo's Room. Lots of junk all over the place due to loads of unfinished projects; contains a bunch of Sonic and Nintendo stuff in the gaming corner of the room; there is a mirror that makes every object in its reflection rainbow coloured (except for the walls, floor, and ceiling). There are rumors that this mirror's power can be harnessed to make a gun that shoots rainbows. Level 13 - Storm's Floor. Level 14 - Starship docking station. Styled similar to Star Wars. Currently, Noghiri, Extreme, Cirevam and Lair all have fleets parked in this space. Level 17 - Office of Recorded Speeches and Quotations. Level 18 - The office of Katatonic717. Level 20 - Barracks Level 21 - Barracks Level 22 - Barracks Level 23 - McStudz's Quarters. On the floor, there's a bunch of stray bricks, paper, and half-finished Cups-o-Ramen, as well as a TARDIS that he's been building. There would be a private elevator to Sector G7, so if the need ever arises, I'll be down there in a flash to rip everyone present a new one. Level 24 - Barracks Level 25 - Rock Raiders HQ, serves as a place to create all the buildings, raiders and vehicles. Level 26 - Upload Center. Level 27 - Download Center (currently being deconstructed). Level 28 - Gallery (currently being deconstructed). Level 29 - Minecraft Portal. Level 30 - World of Text Portal (currently uninhabitable). Level 31 - Modification Testing Area. Level 32 - Software Documentation Throwing Room. Level 33 - Woman's Bathroom. Pretty much unused. Level 34 - Leaving Member's Signout Station, Dead (Banned) Member's Graveyard. Level 35 - RockmoddeR's House Level 36 - Forum Games (It's back, baby). The toilets were originally located in just this spot. Level 37 - Webcomics Printing Room. Level 38 - Karsten's Workbench (Locked Down). Level 39 - Biology Department, serves as a place to examine Planet U monsters. Contains a pen with all know species. Level 40 - Teleporter, unknown location. Research here is being conducted by a subsidiary of Aperture Science. Level 41 - Brickonium Refinery, a dangerous location where raw Brickonium is processed into harmless bandwidth. Level 42 - CireLabs, a place where Cirevam can practice his magic, specifically waving light and milking shapes, as well as creating Infomaniac references. Guarded by CireCams. Level 52 - Alexpanter's chocolate storage, along with a rambling Packard Bell PC which runs Crysis surprisingly well. Level 66 - MAGARE Tracking Station, Action Station's DEFCON Planning Station. In the event that "Action Stations!" is declared, the appropriate DEFCON level is selected, and the war council meets up to discuss and take action against spam attacks. Level 67 - Closed down. Sometimes the sound of explosions, heavy artillery and atmospheric electronic music can be heard inside. Level 68 - Closed down.Sometimes the sound of explosions, heavy artillery and atmospheric electronic music can be heard inside. Level 69 - Lord Zakadia Private Crypt, created and maintained by McStudz. Level 70 - Leased by Incom Technologies for future collaborative projects. Level 71 - The mandatory Noodle Location/Forbidden Area/"That Place" that is sometimes mentioned but never actually seen. Level 72 - The Hiatus room. All members and projects on hiatus live here. Level 80: The speedrun section. This floor contains everything about speedrunning. All the records (and the previous ones) hang on the wall, and there is a cinema where you can view all the evidence-movies and discover some tricks for your own attempts. Dusty due to low amounts of people visiting. Level 88 - Pheonyx's Lab. Accessible only by a resonance door, where he experiments with Einstein-Rosene bridge generators, flux capacitors, Zero Point modules and DeLorian DMC12s, among other things that can rip holes in the space-time continuum. Level 322: The public/private office of Sonic322. Level 323: 1 single room that is a giant amalgamation of every level and zone of the Sonic Universe combined into the hardest most difficult racing track ever devised for a hedgehog. Level 399: Former office/research facility of Professor Gerald Robotnik. Current office/research facility of Dr. Ivo Robotnik. Level 404 - Doesn't sppear to exist, but was last recorded as Nishliau's Time Rift room. It contained a very unstable time crystal that sent him into a random point into the future. The crystal is linked to him, and became so shortly after the passing of gmjab, however its effect kicked in after moving office buildings. Level 717 - Trianglular Minifigure Floor, owned by Le717. Level 718 (Sector G7) - Brony Community Outreach. Known as "Bald Spot". Level 775 - Alcom's primary suite Level 776 - Dropship Hangar Level 780 - Mech Hangar Level 999 - Button to the level is missing and the elevator will always skip this room even if you press 'immediate stop' while travelling between levels 998 & 1000, no-one knows why. Going past Level 999 the sounds of a Rickroll can be heard. Level 1313 - Noghiri's secret base. Level 1848 The Cheese Room - A room completely and absolutely dedicated to every and all forms of cheese. Level 1991 - apemax's floor. He builds with LEGO plus LEGO, computers and consoles/games everywhere. A big mess basically. Level 1997: Infoderpiac Memorial/LEGO Information Center with FLOWERS and a car or a bike or something. Level 1999 - RocketTheRacer's personal room. Resembles the LEGO Racing champ's personal track from the first game & a drawing desk filled with fanart with the Plush doll of RR on it & a PC with a drawing tablet. Level 1,337 - The 1337 Club, a Cyber Cafe for RRU members only. Level 3,141 - Potato storage. Level 3333 - s0d3rb3rg's place. Loud music is playing when he's working. Level 4,998 - Kitchen for mandatory 5-star steakhouse. Level 4,999 - Mandatory high-altitude 5-star steakhouse. Level 5,000 - Observation deck. Level 8000 - Club Archon, IceHusky2's DJ estate. Come for some mad beats. Level 8001 - IceHusky2's hotel. Level 8002 - IceHusky2's personal estate. Invite only, locked out otherwise. Level 8989 - Cap't Rex's Penthouse. Has a rollercoaster, nether portal, hia own personal squads of Magna-guards, 2 legions of battle droids to protect hum, 3 squads of the 501st troopers under his command, 2 AT-TE's, pet creepers, pet lava monsters, a ventor class star destroyer and a Providence-class destroyer. It also features a huge alien ware pc, huge plasma screen TV and etc. It also features squads of vulture droids/heyena bombers, and ARC-170s to defend him. Level 9001 - Cyrem's Administration Office. Level 9002 - Roof. The initiation process involves taking a brochure from the registration desk and filling out a 900 page form. If a member loses their pen, they are required to obtain another one. After completion of the forms, you must pass multiple tests, all of which take about 20-30 years. Are completion, they receive a drill and a shovel. Anonymouse, when around, always uses up the paper towels, and doesn't replace them. Now guys, I need your help. We have a lot more to write about, so let detail exactly what I need from you: Events RRU has a pretty BLAM!tastic history, and I'm currently writing up a whole series of events that cover LZ leaving, the Brony War, the site being shut down multiple times, and much, much more. That said, I need specific dates, information and more events. If you have anything you'd like to add to the timeline, please comment. Random Facts The Anonymous fact is a random fact. The toilets are a random fact. If you know anything about the building or members or ANYTHING that should be considered a fact, please write it down. The more, the better.1 point -
Lwo/lws Basics With Milkshape
lol username reacted to Cirevam for a topic
As long as you have a program that can import them and can export to LWO or some other format that another program can import, there's nothing special that needs to be done. Milkshape looks like it can import FBX so if the importer isn't broken that would be easy. Just import, fix any materials and other things, then export to LWO. And guys, don't talk about spam posts within a topic. That's just contributing to spam. Report it and pretend like it's not there, because when it's eventually deleted the flow of posts won't be disrupted.1 point -
Scaled 640x480 fullscreen?
Nishliau reacted to breakingspell for a topic
I completely forgot to list the specs of my laptop, haha. I'm running Windows 7 Ultimate on an AMD Phenom II X4 mobility chipset with an ATI Radeon HD 4250, the screen resolution was set to the highest, 1366x768 (the game would automatically resize it), and i had 32-bit color on. When the graphics dialog would pop up, i'd select fullscreen at 640x480 resolution. And thanks for the warm welcome! I'll try to play around with my Catalyst Control center (AMD's graphics utility), and see what i can't do1 point -
RRU Quotes 2008-2013
lol username reacted to Lair for a topic
[7:34:05 PM] Cirevam: But The Empire built things to be intimidating, not effective. [7:34:24 PM] Lair of Rockwhales: the empire didn't exist at this point [7:34:37 PM] Ramius Antillies: AAT isn't Imperial. [7:34:37 PM] Cirevam: FFFFFFFFFFFFFFFFFFFFFFFFFFFFF-*walks into lava* [9:33:12 PM] Lair of Rockwhales: HE COMES [9:33:31 PM] Jamesster: THAT'S WHAT ZALGO SAID1 point -
RRU Quotes 2008-2013
s0d3rb3rg reacted to lol username for a topic
17 Jul Shadow322 Yes or no, is it ok for a straight man to get a mani-pedi? 18 Jul Lair of Rockwhales What did your brother do this time? [12:35:24 PM] jamesster: YouTube mobile doesn't let you reply [12:35:29 PM] jamesster: Or vote on comments [12:35:32 PM] jamesster: Or flag comments [12:35:40 PM] jamesster: Or do anything with comments except read them [12:37:08 PM] Lair of Rockwhales: >don't use mobile >problem solved >now I can stop complaining about how phone computers are so horrible and waaaaaah cry more lair you little babywhale I'm going to run fast and ram into small mobile laser cutters and they go BOOM and I say AHAHA I AM A FAST ROVCKWHALE AND IM RED and bandit says OH NO HOW WILL I EVER RUN FROM YOU MISTER WHALE and I'll say YOU SHALL TURN INTO A GIANT MAN and then OH NO BABIES but BabyWhale and NOM and then AN ENENY ENENY HAS BEEN ANENENIED and AAAAA1 point -
~The Brony Herd~
STUDZ reacted to Cyrem for a topic
Perhaps they are trying not to start a fiery argument? They know how it turns out everytime.1 point -
~The Brony Herd~
STUDZ reacted to Fluffy Cupcake for a topic
A program on CNN made a report on bronies. Yay it wasn't negative. http://outfront.blog...ts-the-bronies/ Also, a pony portal animation just came out recently, McStudz will probably love this (not addressing you dirrectly Studz 'cause you aren't on often enough).1 point -
~The Brony Herd~
STUDZ reacted to Zephyria for a topic
Friend linked this to me. I lol'd. And no, we're both bronies, as is the person who made the image. So we weren't making fun of the community.1 point -
~The Brony Herd~
STUDZ reacted to Fluffy Cupcake for a topic
I use Riivolution. Anyway.. this isn't a modding Wii discusion, so, back on topic! Calvin and Hobbes pony edition!1 point -
~The Brony Herd~
STUDZ reacted to Sonic322 for a topic
I found this on know your meme. It's pretty funny. Come on guys. We need to get more posts in this thing.1 point -
Rock Raiders United Incorporated Research - Help Wanted!
STUDZ reacted to mumboking for a topic
@Phoenyx Floor 88 would be more fitting for the Deloreans and flux capacitors.1 point