PDA

View Full Version : HCE And Eight Is Great



Kornman00
August 8th, 2010, 05:22 AM
Today's Open Sauce update is brought to you by the number Eight.

Intro:
Open Sauce. Yelo/Battery. Project Yellow. All of these names are what some, if not all, of you have come to know as the series of Halo engine modifications done by Kornner Studios (formally Kornner Software). A tribute to the countless players who have made multiplayer damn fun and damn funny. A tribute, to the various Halo makers across many platforms for making such a fun game, and fun story. They move on, creating new games. We stick around, get the lay of the land our sandbox is in, and iterate on what they couldn't in their schedule. They're bound by game development deadlines. We're bound by free time, our own sanity and something about copyright...walls or maybe it was calls. Which ever it is, we tip toe across tight ropes making sure the holes in these walls, or the calls we ignore and let ring throughout the day and night, don't harm the original product these copyright holders spent countless man hours to make.

Rules just don't end at the law. There are also rules defined by the engines which power these great games. They have strict standards and formats. Guidelines that we as engine modificationers (why yes, I did make that word up) must learn and respect in order to keep with the feel and mood of the playground which we've been given the privilege to play in. If we don't, that privilege can be quickly revoked. You don't come into a new school and decide to rework the entire playground the very first day. You have to earn your right of passage by those who were there first. The Sandbox of Life in a way.

Details:
Now that we got the formalities out of the way, how about we get down to business?

Officially, Open Sauce has been skitting around as a Release Candidate for a year now. Hard to believe actually. I really didn't want it to go on this long before something solid came out but it's kinda hard to develop for a game when your machine flops and you're stuck waiting until the summer for new hardware to come out for your new development machine.

However, we now see Open Sauce at 2.5. The old features have had time to mature but there are also some new and really awesome features you can now throw on your plate. Really, this should be an entirely new version of OS...but I feel the original OS2's RC was weak and buggy (hey, it was a RC!). Not to say this new version won't have some bugs, but you can rest a little easier knowing that I and a couple others spent many hours pushing the code through tests. Testing is no joke when you're modifying a closed source system. You're confined to how it starts up and the resources it uses. So when it comes to say the game side of things, performing tests can be a major pain as you may find a bug and think it's a simple code change. You exit the game, recompile then start the game back up yet again. There really is no way to perform true unit tests with modifications like OS. Maybe that's something the Kornner should look into changing, hmmm...

So what's new? Well I'm sure most of you will get very happy at the news that we're v1.09 complaint. The upgrade for that took about two solid days (since I had to muck around in both the client and dedi server) but I think it was time well spent in the end. The Halo 1 SDK in OS still includes the v1.08 support but after this release I'm not making as much of an effort on updating the backend memory layouts for that build of the game.

Continuing, I've also spent a long time now working with FireScythe getting his Postprocessing rendering into a solid system that plugs into OS and that also follows most of the standards it goes by. The Postprocessing system can now have its resources stored in some pretty robust tags or even dynamically used by shaders placed in your profile's HaloCE folder (where your game saves are stored). The latter allows you to develop and test your tags in game before you lay them down into tags. You can thank FireScythe for keeping up with systems that work in both tags and dynamically in game!

Another feature we've touched up on: User settings. I wanted this release to be more UAC (I don't know about you guys, but something I actually use) friendly by storing 99% of its data in the HaloCE documents folder. We've also spent time incorporating the use of XML for our settings storage instead of plain-old-text. I was really hesitant of this at first, as it adds yet another library dependency to the SDK. After some initial hiccups and swearing though, we got it working, and it was nice. Really makes things a lot easier for not only us, but the end-users too.

We've also added HEK configuration settings. This is a really awesome new feature to the toolset that I think many people will really enjoy. Are you tired of having to copy 3+ EXEs and countless DLLs to different folders where you have stored countless different tag folders? Don't you wish you could have a single storage location for your executables and then for your various tags folders? Well, now you can! You can setup an XML file which specifies an infinite amount of different "profiles". These profiles define the root directory of your tags folder (ie, "Halo Custom Edition"), plus the tag folder name (ie, "tags" or "tags_h2ce"), the data directory, and maps directory. Basically you can now control where everything gets fed in or shat out.

This doesn't stop at game data folders either, all debug logs are now stored under your HaloCE user profile in the OpenSauce directory along with your OS user settings. All in one place now. Every OS HEK exe will use these file locations no matter where the EXEs themselves actually are. Less clutter on your harddrive! Of course, with the new settings introduced in the OS HEK exes, you should only need one set of HEK exes ;). Debug.txt will no longer bitch at you about the missing beta.map files at start up. This makes it so that the debug log won't be modified unless there is something that actually needs to be written (ie, open edges!).

When it comes to Guerilla/Sapien, you can also specify the profile you want it to use from the settings, via a command line argument much like you use "-console" for the game's exe. This enables you to instead use multiple shortcuts to the same set of EXEs but with different profile names that define what paths to use. So you can have a shortcut to G/S that launches your stock tags environment. Or a shortcut which launches your mod's tags environment. Like the Energizer bunny, it keeps going. Tool however can't specify a profile path since it's a command line application and expects the first argument to be a tool command. So instead, it will use the "default profile" that is specified in your editor settings. If you don't specify a editor profile to use in G/S's command line, then they too will use the "default profile". If no default profile is selected, then no paths are overridden and the EXEs will behave with the same paths as what the stock HEK uses.

Speaking of Sapien, we also know that some of you have multi-processor issues with it (Vista and higher). Well you can stop manually setting the fucking affinity as you can now specify the affinity that it will use in the editor settings! I know, I know, you didn't get us anything. All we expect in return is that you use these new tools and features to make awesome new maps. Or spruce up existing maps :).

'Wait', you ask in a daze, 'I thought OpenSauce was mainly a game extension?'. Fear not citizen, we've also been improving on and adding to the list of things you can do with the game engine too.

People have been doing some pretty neat things with multiplayer maps and scripting over the years. However, many of these neat things have been powered by some sneaky tricks people have thought up. We wanted to introduce some new functionality that would make some of these tricks less tricky and more empowering. We've introduced new functions that let you query various properties of players, units and weapons. You're not just limited to players either, you can work on a team basis too. Querying properties isn't the only thing you can do with players either. There are a few new functions which let you manipulate them as well.

Slap Chop (http://www.youtube.com/watch?v=UWRyj5cHIQA) (You'd really love our nuts)

Of course, not all features can make it off the cutting board. We had to Slap Chop a couple ideas from being finalized or making it into the SDK. One of them was Synapse. Synapse was/is a server tool that was planned for HaloCE that was mostly focused around very detailed stat collection. These stats would be gathered across the entire game. All players who entered and left (and came back again) would have their stats collected along with any shenanigans they performed during the match.

Should be easy right? Well, more or less. You see, before the idea to try Synapse out for HaloCE (it was originally theorized for H2V, back when the game had some hope still left) came about I was working on adding saved films to OpenSauce. It couldn't be done on the scale of Halo3 of course, but it'd at least be something (which is better than nothing). I had a pretty good collection of engine data on how I could go about doing it. Eventually I hinted at the idea to someone, though I forget where. From there, the idea was that an entire HaloCE match could be recorded with all of the events that were fired during the match. From there, the event stream would be compressed and sent to the server for processing. Then, an internal tool would process the entire "film" generating an intermediate file which Cerebrum, the web server to the stat system, could parse and then display to users.

With this system we'd be able to keep records of past games and in the event that we came up with new formulas or more stats, we could go back and re-feed the game data for processing. Other benefits would have come from this, to include detecting cheaters (both in automated form and by manual inspection). In which case the game would then be considered invalid. List goes on, but I'm sure you're getting the idea.

Bringing back the "gravity gun" that was seen back in 2005/2006 was also considered. However it just wasn't possible in the time frame we had left. I was even talking to someone who said that if I could bring this back along with the new ideas I had for it, they'd totally make Forge World for CE (not a remake but a map tailored to CE). Coupled with some of OS's other features (ie, mod-sets), and this Forge World CE would have been pretty powerful and data friendly.

When it's done.
We're just at the tip of the iceberg. By the end of this month you'll get to experience the full monty of the SDK. Doesn't matter if you're an engineer, a modder or a casual player. You'll find something of use in Open Sauce that will greatly enhance how you enjoy your Halo experience.

Note: Yes, I know this is just a wall of text. I'm out of town right now and don't have the ability to take screen captures and/or video. Some things will just need to be experienced in the end anyway. That is what Open Sauce has been about mostly: better user experience.

L0d3x
August 8th, 2010, 05:38 AM
Sounds interesting.

Will there be any significant enhancements for people who mainly develop single player maps?
Would've loved to see the Gravity Gun back though ='(

Kornman00
August 8th, 2010, 09:46 AM
Off hand I can think of at least one thing: there are the bsp enhancements which allow you to do some dynamic changes all within one bsp. Most of the new additions can be applied to both campaign and multiplayer. It'd really be up to map makers to explore what they can do with the new scripting additions, postprocessing, etc just like they did with the HEK when it first came out. That and if they have an engineer, they can do whatever they put their minds to.

Hunter
August 8th, 2010, 11:25 AM
ITS ALL LIES!!!

Sounds awesome though :P Can't wait.

One question about the gravity function; can the gravity be set lower in a certain volume so space maps could be made? So there is gravity indoors, but none outdoors?

Con
August 8th, 2010, 12:12 PM
If the gravity function affects everyone and everything, then no. If there's something to adjust gravity for a single player (korn mentioned player properties) then it shouldn't be an issue.

Kornman00
August 8th, 2010, 02:43 PM
gravity scripting works EXACTLY like it does in Halo 2 now (via functions, not a global anymore)

You wouldn't be able to do zone specific gravity in MP maps. Reach will have it though :downs:

Sean Aero
August 8th, 2010, 08:27 PM
Of course, not all features can make it off the cutting board. We had to Slap Chop a couple ideas from being finalized or making it into the SDK. One of them was Synapse

What was the main reason for the drop? I'm curious because reading all the features of this tool kinda made me drool.

Kornman00
August 9th, 2010, 03:12 AM
Development time. I'm only one person, and I only have so much free time to dedicate to this project. I have to allocate time inside that free time for R&D on 5 EXEs (when it comes to Halo 1 at least), engineering a handful of libraries (some of which modify these programs in real time), then testing all of the these things.

Hunter
August 9th, 2010, 06:56 AM
gravity scripting works EXACTLY like it does in Halo 2 now (via functions, not a global anymore)

You wouldn't be able to do zone specific gravity in MP maps. Reach will have it though :downs:

So, let me clarify; if the gravity value is changed then it affects every player? Could you not allow it to affect just one player?

L0d3x
August 9th, 2010, 08:17 AM
Hate to bother you with a potentially nooby question, but with Open Sauce, would/could it be possible to edit certain fields within tags ingame?
Cause that would be lovely.

Kornman00
August 9th, 2010, 10:37 AM
Programmatically you could.

So, let me clarify; if the gravity value is changed then it affects every player? Could you not allow it to affect just one player?
It's the game's global gravity, so no. Halo 1 never had object-specific gravity scaling.

STLRamsFan
August 9th, 2010, 10:38 AM
Good stuff man, looks like I might actually have to get off my ass and reinstall HCE to try this.

L0d3x
August 9th, 2010, 10:50 AM
Programmatically you could.

Was that in reply to the field-edit question?
I'll assume so. That is awesome, and offers actual hornet physics to be realised if it's true.

Kornman00
August 9th, 2010, 11:24 AM
Yeah it was. You'd need to have a programmer obviously.

Dwood
August 9th, 2010, 01:18 PM
So the entire OS is completely updated to 1.09, not just the addresses that are used from the get-go?

Kornman00
August 9th, 2010, 04:53 PM
All addresses that have been updated. Not all have been tested as not all are used (ie, campaign level extensions). But all which are used by active systems have been tested.

Kornman00
August 16th, 2010, 05:50 PM
Ugh, sapien isn't playing nicely with what we were going for. Sadly, sapien will not be able to make use of tags folders in different root directories, however tool and guerilla still will. Ex, if sapien is located at "C:\halo1\sapien.exe", but the tags folder is located at "C:\halo2ce\tags", it won't be able to access those tags.

Sapien will however allow you to use different tag folder names as long as they're in the same directory as sapien (ie, ""C:\halo1\"). So you can three different tag folders in your HEK folder: "tags", "tagsh2ce" and "tags_BLAM" and then have profiles for each of those in your settings. From there, you can make shortcuts to sapien which then specify which profile you want sapien to use via the command line.

Something is better than nothing as they say :\. At least tool and guerilla will still work with tag folders existing anywhere on your computer.

CrAsHOvErRide
August 16th, 2010, 09:29 PM
I fixed mah Sapien a long time ago. Just nop'd a whole function that caused the dual-core error. Not the right way of doings things but I lol'd.

Kornman00
August 29th, 2010, 06:03 AM
I just lost one or two days off my schedule. I was working on getting the codebase synced with the source control yesterday. Found out that there was an update to my SC's client so I updated that. Windows also told me it had a couple updates too so I figured since I had to restart my computer I'd perform the next sync after I rebooted. Well I reboot and W7 fails to launch (http://www.thewindowsclub.com/fix-stop-0x0000006b-process1_initialization_failed-blue-screen). I tried a couple different things but I didn't google the BSOD error. Instead I just Installed a new copy of W7 onto a different partition and started the configuration process all over again (decided to see what the BSOD was all about this morning). So I've lost at least a day with having to redo my computer :\

Just like cellophane and having to get a rope, it's always something (http://www.youtube.com/watch?v=l81W-jcOuj0) :ugh:

CrAsHOvErRide
August 29th, 2010, 10:08 AM
Don't worry. Recently I had bad sector on Windows DLLs. Was fun to fix it with no Windows CD, no Recovery and no Ubuntu.

Kornman00
September 16th, 2010, 05:25 PM
Head deep in Reach right now. Computer breakage kinda borked up the rest of my schedule. But not my reach schedule. reach reach reach

Dwood
September 19th, 2010, 03:17 PM
Head deep in Reach right now. Computer breakage kinda borked up the rest of my schedule. But not my reach schedule. reach reach reach

You bring your computers to their knees bro.

Kornman00
September 27th, 2010, 11:55 PM
Well I'm out of the loop for the next week, out getting stuff done. I don't think anyone has been paying attention to our google-code hosting (http://code.google.com/p/open-sauce/updates/list) but we've (FS and I) been using it with our source for a while now. Still doesn't have the entire OS source tree under it (missing a couple future additions, ie BlamLib) but it should give OS developers something to look at and even use some what (doesn't include the new OpenSauceIDE or OS_tool, etc builds).

Dwood
September 28th, 2010, 01:26 AM
Wow, bitterbanana is on there... whats he doin?

Skarma
September 28th, 2010, 02:42 AM
I'm sure his name is just still there from when the project was created years ago. He doesn't seem to be active anymore. Very cool stuff guys, I love SVN! I know to commit changes you need to be a project member, but say someone wanted to contribute (which I do), what do they need to do? PM you the information or haggle you for membership?

Kornman00
October 5th, 2010, 11:00 PM
bitter is on there because I nagged him to accept the group link. Doesn't mean he'll ever commit any changes...but doesn't mean he will either. He helped a lot in quite a few parts of the development of Project Yellow so I he deserved access even if he doesn't use it now or ever.

Well, it's not actually SVN. It's Mercurial which is aimed more at distrib-development which is something I thought fit OS's development more than how SVN or (*shiver*) CVS works.

FS is a contributor because I know what his goals are, I know that he's gotten the hang of OS's ropes and the standards used (more or less). If I were to allow anyone else more than read access I would first need to know what their intentions are (since this is the root base of OS and thus everything HAS to work together and without fault) and know that they will stick to the standards/formats that OS uses. I had to help FS learn to run which I didn't mind at all but I really don't have the time to do with anyone new. So any new comers would basically have to create a proposal of what their intended commits are to the source tree, why they're including them and what true value those commits will give to users (either other devs, modders or end users).

Hate to make it sound like business and red tape but right now the goal is to just get OS2 finalized and tested for any last major bugs. I know source control "trunks", "branches" or what not can be used for new and untested content which can then later be merged, but I really don't want to complicate the source tree anymore than it has to for not only FS and I, but also any anonymous browsers. I know I'm savvy with tree branches, not so sure about FS, so I don't want to initiate a fire which I may not be able to control the burn of.

Skarma
October 12th, 2010, 10:12 AM
I'm having trouble downloading a copy of the repository. I am familiar with google code and client programs, but it is just not working and throws me an error. I am using RapidSVN and I use to use TortoiseSVN, but it was having shell problems. I go to export and type in the directory: https://open-sauce.googlecode.com/hg and choose my output path and BAM. It tells me: "Error while refreshing filelist (Server sent unexpected return value (405 Method Not Allowed) in response to OPTIONS request for 'https://open-sauce.googlecode.com/hg'). Google Code is doing server maintenance on Thursday, but I don't see how that might effect things right now.

Also is OpenSauce still supporting of other blam games? Would be cool to see the solutions/projects already prefabricated in the trunk. I was really interested in REing other games, like Stubbs the Zombie but I don't even know where to get started with setting it up with OS. I know you said something about shared files in your Wiki, but I only see some really basic files like macros and type definitions in shared. Is there more to this? What are the details of BlamLib?

Kornman00
October 13th, 2010, 02:38 AM
We don't use a SVN based repo, we use a Mercurial powered one, so you can't sync using SVN tools. Google code has resources you can hit up for accessing Mercurial repos with whatever OS and/or editor you use.

The other blam-based games I've written code for is very specific. The only other engine extensions I've written are for Halo 2 Xbox and H2V's tools (so there are no source projects for a Stubbs game for any platform). Neither have been synced yet as the current goal is just to get Halo 1 up and in running order. The H2X codebase also makes use of a generic Xbox1 "module" (aka, "trainer") system which I'm still finalizing the standards for since I'm wanting modules to be loadable by future Xbox (ie, dxbx or cxbx) emulators without having authors reprogram/relink their shit. BlamLib (which is 95% C#) has been positioned for future syncing to the repo but I'm still redacting some parts of the code not meant for public use and also adding some test code to help illustrate its usage.

Shared files are typically just there for generic code sharing or for project sharing. IE, the Halo1_CE and Halo1_CheApe both use shared project source files. This is so I don't have to fuddle around with setting up some honky tonk LIB which only complicates the build process that much more. Shared files are just that, shared to keep out dup'd code and to allow easier compiling/linking of relate-able code.

FireScythe
October 13th, 2010, 06:55 AM
TortoiseHG (http://tortoisehg.bitbucket.org/) is a shell based tool that you can use to interface with the code repository.

Shock120
October 13th, 2010, 07:19 AM
The other blam-based games I've written code for is very specific. The only other engine extensions I've written are for Halo 2 Xbox and H2V's tools (so there are no source projects for a Stubbs game for any platform). Neither have been synced yet as the current goal is just to get Halo 1 up and in running order. The H2X codebase also makes use of a generic Xbox1 "module" (aka, "trainer") system which I'm still finalizing the standards for since I'm wanting modules to be loadable by future Xbox (ie, dxbx or cxbx) emulators without having authors reprogram/relink their shit. BlamLib (which is 95% C#) has been positioned for future syncing to the repo but I'm still redacting some parts of the code not meant for public use and also adding some test code to help illustrate its usage.
It's really bad, the reason why they would find H2X emulation hard, I am missing Xbox1 titles namely Halo 1 (awesomez netcode) and H2X.

Halo 2 is build using link-time-optimized libraries, making it near impossible to automatically & generically detect symbols. As a result, Dxbx will not support those type of titles. Your best chance would be with Xenoborg (the low-level Xbox1 emulator, being worked on by Blueshogun96, albeit at a slow pace), I'm afraid. (http://forums.ngemu.com/dxbx-official-discussion/134248-screenshots-dxbx-6.html#post1906911)..
Great job kornman00, as always. :realsmug:

Skarma
October 13th, 2010, 10:41 AM
We don't use a SVN based repo, we use a Mercurial powered one, so you can't sync using SVN tools. Google code has resources you can hit up for accessing Mercurial repos with whatever OS and/or editor you use.
This is strange. I am guessing this is an option in Google Code? I have never had a problem with the tools I am using and they have always synced for the past 2 years with the projects I have been a part of...

I know I saw a "Stubbs" folder in one of your OpenSauce demo videos, don't whack around the bush lol!!!

Kornman00
October 13th, 2010, 02:03 PM
Like I said, we don't use SVN. We use Mercurial. They are two totally different SCCs. Google code allows you to use SVN or Mercurial as the SCC back-end of choice. I picked the latter.


Great job kornman00, as always. :realsmug:
Well when you use the methods they use for symbol detection, of course it's damn near impossible. Complicated problems, like LTCG, require "smarter" (as in "dumb" AI vs "smart" AI, not as in "their system is stupid") detection algorithms which track more detail than just a series of 20 or 32 bytes. There is so much more detail which they can gather and track from the SDKs for symbol detection that they're not using with their system.

If you happened to notice, I said the Xbox1 module system is made for any targeting any game. So a Halo1 or Stubbs module could be written with it.

Great post Shock, as always!

Skarma
October 13th, 2010, 11:14 PM
Ok so I finally got a client program working. I cloned a fresh copy of the source tonight, didn't add any code, compiled a release build and threw the d3d9.dll in my Halo CE folder ( I am running 1.09 ). Halo crashes on startup. I ran Halo through OllyDbg and it is crashing somewhere in the IDirect3DDevice9::Reset() call in the engine.

Edit: After commenting out the post processing system, the crash went away. I think it's because I don't have the required files loaded and used by this system like the shader files, which were not included in the google code project so how was I suppose to know? This shouldn't crash Halo though, there should be a check in place for this kind of thing.

FireScythe
October 14th, 2010, 12:17 PM
I've just pushed a fix for this. I hadn't put a null pointer guard around the gbuffer debug effect object after I moved it into its own class. Sink ur sauce.

Skarma
October 14th, 2010, 01:02 PM
GREAT! Works like a charm now, thanks FS! I also noticed Kornman did not put any null pointer checks for his menu resources. It will probably never crash because of the way it's setup, but it can happen so I would probably recommend adding them just in case and for good programming practice. Oh and I seen that the TextBlock class is created with the new keyword, but isn't it a memory leak when not deleting it before the game is closed? Thanks guy!

Edit:

Tip for other developers, to switch Halo versions, go to Common -> Platform.hpp and at line 40 and 41 is where you would uncomment the version you are building OpenSauce for.

Skarma
October 16th, 2010, 12:51 PM
I have a small request to add some math functions to BaseBlamTypesPc. Here are a few I have written into OS and use currently. I am working on an angle distance one and I'm sure there will be other very common ones used in 3D math:

real_point3d(applies to 2d also without z):

API_INLINE real Distance(real_point3d& rp) const
{
real_point3d d = {
this->x - rp.x,
this->y - rp.y,
this->z - rp.z
};

return (real)sqrt( d.x*d.x + d.y*d.y + d.z*d.z );
}

real_vector3d:

API_INLINE real Dot(real_vector3d& v) const
{ return (real)( this->i*v.i + this->j*v.j + this->k*v.k ); }

API_INLINE real_vector3d Cross(real_vector3d& v) const
{
real_vector3d cross = {
this->j * v.k - this->k * v.j,
this->k * v.i - this->i * v.k,
this->i * v.j - this->j * v.i
};

return cross
}