Installation:
-extract the zip into an empty folder with read and write permission
-start the ImageShaper.exe
This tool was created to overcome the flaws of SHP Builder, which are
-very slow import (UI freezes during import)
-impossible to combine different render passes during import
-inconvenient palette management, which needs a custom palette for each import that wants certain colors to be ignored (e.g. glowing colors, non-remap reds etc)
-impossible to import already paletted images. SHP Builder still applies it's own color conversion, which messes up different palette colors with the same RGB value (e.g. a glowing pixel with color #240 is changed into non-glowing color #15, non-remap reds changed into remap).
With this tool you can convert up to 3 different render passes in one operation into a single SHP, while using only a single palette file.
This palette gets assigned to each image with possibly different settings, like ignore glowing colors.
For those who like comparison numbers:
import of 576 files with 200x200 pixel size
SHP Builder: over 3 minutes
Image Shaper: 25 seconds
How to use
Drag and Drop files on the datagrid or use its right-click context menu to load image files.
Assign a palette to the images.
-right click on the palette and load a palette
-right click on a color which should be ignored during the conversion into the palette
-the color is marked red
-hold-click on that color and drag the mouse to other colors, which should have the same "ignored" property
-select all images, right-click on the selected images and set "Set palette", to assign this palette with these modifications to the selected files
-clicking again on a single image, the preview will show it already in the final converted state
-drag and drop (or load) a second set of images in the 2nd column, which should be combined with the respective image in the first column
-select the images in the 2nd column, do the necessary changes to the palette
-in this case, dark pixel should be converted into transparent. Thus "Make Transparent" is used instead of "Ignore Color", as the last one would force dark pixel to be converted into the only left colors, which are the glowing colors
-after the palette changes are done, apply the palette to the selected files
-this way we got rid of dark anti-aliased pixel and have only the glowing pixel left in the right palette color
-in this example another set of a images of glowing pixel from a non-anti-aliased render pass is applied to the 3rd column
-these got again a certain modified palette assigned
With the non-antialiased glowing pixel...
...the anti-aliased glowing shapes...
...and the unit without glowing pixel, ...
...a click on the row to the left, shows the combined images in the preview.
Finally a click on [Start], will start the conversion of all the files into a single SHP
a few final adjustments can then be done in SHP Builder
Version 01.00.00.00
-first public release
Version 01.00.00.02
-(bugfix) compiled as x86 to match the x86 MagickLibrary and prevent mismatch on x64 systems
Version 01.01.00.00
-(update) SHP creating routine implemented, which allowed to get rid of the big ImageMagick library and the limited ShapeSet tool.
-(update) SHP reading routine implemented, which allows to load SHPs and have the frames listed for further resaving and modification.
loaded SHPs are not color converted again and read/shown as indexed palette image. Color Conversion methods are not applied on these.
-(update) Radar color setting added, allowing to modify/set each frames radar color. Used by TS/RA2 only for Overlay SHPs like tiberium/ore.
-(update) Added option to create an image for each processed frame.
-(update) Added option to prevent wobble bug in TS. This fixes turreted SHP units and prevents them from wobbling 1 pixel up down when rotating the turret or when using a moving animation.
-(update) Options to set the compression for each frame.
-(update) Hovering the mouse over palette indexed pixel in the preview image, highlights the used color of that pixel in the palette
Version 01.01.00.01
-compiled as target platform "any CPU"
Version 01.01.00.02
-focus is set back to the datagrid after SHP conversion process is finished
-SHPs with wrong compression bit set are loaded anyway
-last used settings stored/loaded from ImageShaper.ini
Version 01.01.00.03
-(bugfix) bad ini values for position and size don't crash anymore
-(update) simple palette manager added, which keeps track of different palette settings
-(update) image files only point to a palette from the manager and don't keep a full copy of it, thus hopefully reducing memory usage
-(update) palette edits are instantly applied to all files sharing the currently selected palette
-(update) performance of deleting files/frames improved
Version 01.01.00.04
-(update) images/frames store only an int index to the used palette from the palettemanager, thus reducing memory usage per image/frame
Version 01.01.00.05
-(bugfix) palettemanager crashed on images with unassigned palettes
Version 01.01.00.06
-a few adjustments on the class for the internal stored images/frames
-progressbar is showing the current progress of the imported files
Version 01.01.00.07
-(bugfix) loading palette indexed images was not supported. Now loaded directly without any color conversion.
-(bugfix) SHP frames loaded in the 2nd or 3rd imagelist caused an exception
-(update) images with 32bpp ARGB, 24bpp RGB and 8bpp palette indexed color formats are directly supported. All other image color formats are converted to 32bpp ARGB before processing image in color conversion.
-(update) added options to optimize canvas size
-(update) BitFlags added
-(update) option to calculate average radar color added (ignoring all transparent background pixel)
-(update) Routine to identify SHPs changed to be less strict. Previously anything with a BitFlag value of >3 was discarded as invalid SHP.
Version 01.01.00.08
-(bugfix) optimize canvas failed when empty frames were included
-(bugfix) when using "RLE_Zero" compression, the bitflag wasn't set in the SHP
-(update) last used average color checkbox value stored in ini
-(update) when enabling "optimize canvas", "keep centered" is enabled by default as well
-(update) option added to use a custom color as transparent background color during color conversion
-(update) option added to copy only the transparent pixel when combining this image with a base image
-(update) changes in the "Image/Frame Settings" are instantly applied to the selected images/frames
Version 01.01.00.09
-(update) create images expanded with SHP(TS) file format, allowing each frame to be saved as SHP
-(update) create images filename can now use an asterisk *, to keep the original filename for the single frame
Version 01.01.00.10
-(bugfix) RLE-Zero encoding algorithm crashed when the encoded result data was bigger than 2 times the uncompressed data (RLE encoded worst-case is 3 times the size as uncompressed)
-(bugfix) imported files ignored the "fixed Backcolor" setting
-(update) faster file import in datagridview (now refreshes only once after import is complete)
Version 01.01.00.11
-(update) JASC Palette format supported
-(update) loading files now done in a separate thread to prevent/reduce freezing interface
-(bugfix) when loading multiple SHPs, an empty cell was added after each SHPs last frame
Version 01.01.00.12
-(update) if preview window is focused, ctrl+c copies the image into the clipboard
-(update) Split result added, which allows to split the frames evenly into multiple SHPs.
Version 01.01.00.13
-(bugfix) RLE Encoder fixed (again). Now resizes array during encoding if encoded data exceeds the preallocated array size.
Version 01.01.00.14
-(bugfix) RLE Encoder fixed (again).
Version 01.01.00.15
-(update) internal Drag & Drop added to the datagrid cells. If a cell is empty, the d&d value is set, otherwise a new row inserted. Existing values are not replaced!
-(update) [File] menu added to menustrip, which offers functions to save and load a project. A project includes the data from the datagrid and the complete palette setup.
Version 01.01.00.16
-(update) "Load from Clipboard" added to the Datagrid right-click menu. The clipboard image is saved in the Temp subfolder.
-(bugfix) Copy, Cut, Paste in Datagrid right-click menu fixed
-(update) datagrid ensures a single empty row at the end
-(update) preview window moves to front as well, when program gets focus
Version 01.01.00.17
-(bugfix) annoying windows warning sound removed when adding files
-(bugfix) "combine transparent pixel" is applied only to the image in the column left of it
-(update) if multiple cells are selected when adding files, the user is asked if the files should be duplicated into the cells
-(update) the audio file SHPfinished.wav is played when SHP conversion is done
Version 01.01.00.18
-(update) support for RGBA JASC palettes
-(update) Euclidean color conversion with Alpha-channel support
-(update) [Options] menu added
-(update) "#Worker Threads" setting moved into [Options]
-(update) "Output Folder" setting added to [Options] to define output folder for SHPs
-(update) support to save/load a palette setup. Select "ImageShaper Palette Setup" as fileformat in the Load/Save Palette Dialog.
-(update) Console command line functionality added. Run ImageShaper.exe ? for help.
Version 01.01.00.19
-(bugfix) Preview shows alpha channel correct
-(update) "Preview Background Image" setting added to [Options]
Version 01.01.00.20
-(update) Preview updates for the selected image automatically when changes to the palette are made (e.g. colors set to "ignore")
-(update) with the DataGrid focused, CTRL+V copies the Clipboard image into it
Version 01.01.00.21
-(update) the Preview isn't locking the access of the shown file anymore. Image files can now be replaced/changed while Image Shaper is open.
-(update) new option "Reverse order of selected cells" added to the DataGrid context menu.
Version 01.01.00.22
-(update) The resuling SHP filename has trailing numbers only removed when operating on image files. When loading an SHP file, the resulting filename is exactly the same.
-(bugfix) in rare cases "opt. canvas" caused an index of bounds error when creating a frame
Version 01.01.00.23
-(update) new format "Uncompressed_Full_Frame" added, which stores each SHP frame in its full size. "optimize canvas" doesn't work for this case
-(update) command line option added to support setting "Split result"
Version 01.01.00.24
-(update) new function "Load and Split Image" added to right click contextmenu, which allows to load an image with frames as panels inside the image.
-(update) new tool "FireFLH Finder" added to menubar "Tools" , which makes it easy to find the correct values for FireFLH, PrimaryFirePixelOffset, DamageFireOffset# and MuzzeFlash#.
-(update) the custom image control (e.g. Preview window) doesn't reset the view/scrollbars anymore whenever the image is changed.
P.S.: i did some extensive testing on different color conversion methods, and the default "Euclidean", which gives similar results to SHP Builders "3D RGB Structuralis", is actually better than DeltaCIE2000.
DeltaCIE2000 has big problems with darker shades and should be used only on brighter colors.
So the SHP Builder color conversion is actually not that bad as mentioned here, it's just slow. _________________ SHP Artist of Twisted Insurrection: Nod buildings
Error message when pressing start:
Could not load file or assembly 'Magick.NET-Q8-x86, Version=7.1.0.0, Culture=neutral, PublicKeyToken=2004825badfa91ec' or one of its dependencies. An attempt was made to load a program with an incorrect format.
The mentioned DLL exist in the folder and started the program by double
clicking the EXE. The source files were PNG files exported from SHP Builder.
For the problem you mentioned on paletted image or retaining the exact
color, I have been using SHP Builder r96 as the last good SHP Builder.
Later versions behave differently. QUICK_EDIT
fixed
the project was compiled as any CPU while the MagickLibrary is x86 only, thus it wasn't working on x64 machines.
It's now compiled as x86 to always match.
Download again from first post to get version 02
Blade wrote:
Not open sourcing it though?
I can send the source code to those who want it, but usually i don't make my tools open source. _________________ SHP Artist of Twisted Insurrection: Nod buildings
shit, ShapeSet has it's limit there.
The message "shp created" simply comes when the program notices the separate ShapeSet process has finished working. I didn't check for errors from ShapeSet.
Ok, i guess this means we can scrap this tool, since ShapeSet is too limited. Fuck.
Anyone interested in writing a simple .NET routine to create an SHP from a set of indexed palette images? I really have no interest in trying to decipher the SHP format.
Best would be something like this
Then i could simply pump the generated frames into the class, without having to use the old PCX file format or any temporary files at all. _________________ SHP Artist of Twisted Insurrection: Nod buildings
Now that would be easier to make SHPs _________________ If you are a MetalHead (Heavy Metal Fan) and don't want to be a metalhead, Just remove your metal ball from your head. �:p .
QUICK_EDIT
it succeeded for 752 frames but not quad of it at 10528 frames.
ShapeSet's limit is at 1019 frames.
Thanks GD. Maybe i'll find some time and interest in near future to find out how to code a simple SHP creating routine. Then these links surely come in handy.
Joined: 22 Nov 2010 Location: Iszkaszentgyorgy, Hungary
Posted: Tue Dec 05, 2017 2:38 pm Post subject:
Lol, well, seems so.
TBH, never used that command myself - even on Linux, I just Wine SHP Builder and/or Mixer instead of dealing with ORA utility.
Heard from Nolt however that the tool keeps the SHP frames as pcx as well - that's cool already. _________________ "If you didn't get angry and mad and frustrated, that means you don't care about the end result, and are doing something wrong." - Greg Kroah-Hartman
=======================
Past C&C projects: Attacque Supérior (2010-2019); Valiant Shades (2019-2021)
=======================
WeiDU mods: Random Graion Tweaks | Graion's Soundsets
Maintainance: Extra Expanded Enhanced Encounters! | BGEESpawn
Contributions: EE Fixpack | Enhanced Edition Trilogy | DSotSC (Trilogy) | UB_IWD | SotSC & a lot more... QUICK_EDIT
yes, palette indexed PCX are stored in the Temp subfolder, as the first step before ShapeSet converts these to SHP.
I would have preferred a more modern fileformat, to get rid of the huge ImageMagick library just for that old format, but ShapeSet seems to accept PCX only.
Is OpenRA supporting PCX files directly as frames? Does Nolt use PCX as well or another fileformat?
Would be surely useful if in future the tool keeps the option to create single palette indexed frame images in the OpenRA supported image format.
I've just started extracting only the relevant code from OpenRA to understand the TS SHP format.
Currently i can load and view already an SHP (huge thanks to the OpenRA devs for that )
and just started writing a SHP writer routine.
Once i can write (TS)SHP files directly, i can get rid of ImageMagick, ShapeSet and create the SHP directly with hopefully no frame limit (except the inherent limit of SHPs, supporting only an ushort number of frames 65535).
P.S. Did i noticed right that OpenRA ignores the radar color of SHPs?
There are the 3 byte values 4 byte after the compression format byte, but the OpenRA reader simply jumps over them
in class ShpTSFrame : ISpriteFrame
Code:
Format = StreamExts.ReadUInt8(s);
s.Position += 11;
FileOffset = StreamExts.ReadUInt32(s);
I'm not 100% sure, but i think
s.Position+=4;
is then followed by the 3 RGB byte values. (IIRC used on tiberium SHPs to hold the radarcolor of the tiberium patch) _________________ SHP Artist of Twisted Insurrection: Nod buildings
Don't use ORA's code for LCW/XORDelta, they never fixed it even after we pointed out the issues with it.
Nyer has been playing around with our code and making a viewer for initially various WW files, now extended to support other similarly built files.
http://nyerguds.arsaneus-design.com/project_stuff/2017/CnC64FileConverter/dev/CnC64FileConverter_v1.2.7_src.rar
Code in it is practically our code converted to C#.
\Domain\GameData\Westwood\WWCompression.cs
He doesn't have reading TS SHP's yet but only compression that has is RLE, i wouldn't be too surprised that WW's existing RLE implementations are it already.
The colors you are referring to are not radar colors, its the average color of all the non-keyed(0x0 index) pixels in the frame, it might be used as radar color indeed, maybe some other uses, we've yet to find the exact code that does it. _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
If i understand it right, LCW/XORDelta is only used for SHP(TD) files, not on SHP(TS) files.
So i can ignore this, since i don't care about TD SHPs. If anyone wants them he can open the generated SHP(TS) and save it as SHP(TD) again using SHP Builder.
RLE_Encoding algorithm could be indeed useful, as this is from what i can see used in SHP(TS) format.
Since i use OpenRA's code to read/show SHPs and it works fine on SHPs generated with the official Westwood ShapeSet tool, i assume that the implementation of OpenRA is correct.
Gonna have to test a few more example though, also some crazy coded SHPs with unusual frames.
ok, i can create uncompressed SHP(TS) files. yay
SHP Builder loads them fine and they are basically exactly the same as the SHPs produced by ShapeSet, if you don't use the -R parameter.
But,
SHP Builder has different compression methods.
Simply sending my image data byte-array into Nyerguds "RleEncode" routine and then writing this, ends in total garbage when viewed in SHP Builder. I've also set the format byte-flag to 3 and the headers look the same.
So, does anyone know how to use the RleEncode routine?
It has for example a bool "swapWords" parameter, though i've no clue if i have to set it to true or false.
"minimumRepeating" is another parameter, where i've no clue if i should set it to 2, 3 or any other value. (tried 2 and 3, but the result is nothing alike the ones produced by ShapeSet with -R parameter and/or SHP Builder)
I almost have the impression the compression is done only on each line, not the entire byte-array of the image, because the hex code of the SHP builder/ShapeSet generated files have some distinctive line patterns left, while Nyerguds RleEncoded bytes are just a block of completely random bytes.
\Edit
after some byte analysis, i think SHP Builder and ShapeSet are compressing zero bytes only, while Nyerguds RleEncoder seems to compress all doubled pixel, regardless of their color.
Thus his routine is not suitable for me. _________________ SHP Artist of Twisted Insurrection: Nod buildings
Why don't you try zzattack's map renderer which should be in C#, and should
lead you to quick result.
didn't thought about that, but i think it most likely has only loading, not writing.
\Edit
yes, only loading.
E1 Elite wrote:
Nyerguds code looks generic and it may not be specifically ready to use as
such for SHP.
There are plenty of source code - stucuk's SHP Toolz, XCC or SHPBuilder
or OpenRA stuff or who else has it.
SHPBuilder is a pain to read, but i'm currently trying to understand and convert its code.
OpenRA has only loading, not writing SHP(TS) format.
I can find on XCC only infos about SHP(TD), not SHP(TS) format _________________ SHP Artist of Twisted Insurrection: Nod buildings
Here's some pseudo if it helps
encoder from ShapeSet
Code:
int __stdcall RLE_40D530(char *source, _BYTE *a2, int length)
{
int v3; // ST60_4@16
signed int v5; // [sp+4Ch] [bp-18h]@13
int i; // [sp+50h] [bp-14h]@8
char *v7; // [sp+54h] [bp-10h]@5
int v9; // [sp+5Ch] [bp-8h]@5
new version released
this one has seen a big amount of changes, mainly direct SHP reading and writing support. So I was able to get rid of ImageMagick and ShapeSet.
Probably the best change however is the option to prevent the annoying TS wobble bug.
Version 01.01.00.00
-(update) SHP creating routine implemented, which allowed to get rid of the big ImageMagick library and the limited ShapeSet tool.
-(update) SHP reading routine implemented, which allows to load SHPs and have the frames listed for further resaving and modification.
loaded SHPs are not color converted again and read/shown as indexed palette image. Color Conversion methods are not applied on these.
-(update) Radar color setting added, allowing to modify/set each frames radar color. Used by TS/RA2 only for Overlay SHPs like tiberium/ore.
-(update) Added option to create an image for each processed frame.
-(update) Added option to prevent wobble bug in TS. This fixes turreted SHP units and prevents them from wobbling 1 pixel up down when rotating the turret or when using a moving animation.
-(update) Options to set the compression for each frame.
-(update) Hovering the mouse over palette indexed pixel in the preview image, highlights the used color of that pixel in the palette
@E1 Elite: i successfully created a 48MB SHP with over 40.000 frames.
So feel free to test out the limits of the SHP format and this tool.
Thanks again to the devs of OpenRA and OS SHP Builder, who helped creating this. _________________ SHP Artist of Twisted Insurrection: Nod buildings
Currently the EXE v1.01.00.00 is not able to handle more than 2gb memory,
it gives out of memory exception.
I modified the EXE to set that flag - 0x96 from 02 to 22 to fix that.
It is able to make SHP upto 22K frames for a dimension I used but with smaller
file dimensions it was able to do more.
The drag and drop dialog box doesn't show more than a specified number of
files based on its size. More files with preview box open, gives exception:
System.ArgumentException: Parameter is not valid.
at System.Drawing.Bitmap..ctor(Int32 width, Int32 height, PixelFormat format)
at System.Drawing.Bitmap..ctor(Int32 width, Int32 height)
at ImageShaper.UC_ImageCanvas.UpdateZoomedPicture()
at ImageShaper.UC_ImageCanvas.SetImage(Image image)
at ImageShaper.Form_DockPreview.SetImage(Bitmap bitmap)
at ImageShaper.Form_ImageShaper.<UpdatePreview>b__a(Object w_s, ProgressChangedEventArgs w_e)
at System.ComponentModel.BackgroundWorker.OnProgressChanged(ProgressChangedEventArgs e)
at System.ComponentModel.BackgroundWorker.ProgressReporter(Object arg)
A command line utility could be an option which would avoid the GUI
limitations. QUICK_EDIT
Currently the EXE v1.01.00.00 is not able to handle more than 2gb memory,
it gives out of memory exception.
I modified the EXE to set that flag - 0x96 from 02 to 22 to fix that.
is that like changing the x86 platform target to x64?
What kind of gigantic SHP are you trying to create? o.O
E1 Elite wrote:
The drag and drop dialog box doesn't show more than a specified number of
files based on its size. More files with preview box open, gives exception:
System.ArgumentException: Parameter is not valid.
at System.Drawing.Bitmap..ctor(Int32 width, Int32 height, PixelFormat format)
at System.Drawing.Bitmap..ctor(Int32 width, Int32 height)
at ImageShaper.UC_ImageCanvas.UpdateZoomedPicture()
at ImageShaper.UC_ImageCanvas.SetImage(Image image)
at ImageShaper.Form_DockPreview.SetImage(Bitmap bitmap)
at ImageShaper.Form_ImageShaper.<UpdatePreview>b__a(Object w_s, ProgressChangedEventArgs w_e)
at System.ComponentModel.BackgroundWorker.OnProgressChanged(ProgressChangedEventArgs e)
at System.ComponentModel.BackgroundWorker.ProgressReporter(Object arg)
A command line utility could be an option which would avoid the GUI
limitations.
what do you mean with drag&drop dialog box? D&D has no dialog. Do you mean the context menu "Load Images" option?
I don't know how that has anything to do with the preview window, since that shows only a single image, so it shouldn't matter how many files you have in the list nor how much you currently load.
x86/x64 is for which compiler to use. Greater than 2gb is a EXE file header
characteristics flag which allows it to handle more than 2 gb address space.
Drag and drop is same as the load images to the list.
i can only compile as target platform "any CPU" on my 32 bit PC.
download again and test version 1.1.0.1 if that works better.
Though i think such unusual huge SHP aren't common or used in the first place.
I doubt the games would even handle/load a SHP that takes up several Gigabyte.
In my tests, 65535 frames of 100x100 pixel loaded fine in the list and also got converted into a SHP.
For the usual or even extraordinary unit that isn't reaching more than 5000 frames max for 32 facings even when having 30 walk, 30 firing, 30 death and 30 standing frames, the current version should work well enough. _________________ SHP Artist of Twisted Insurrection: Nod buildings
I can send the source code to those who want it, but usually i don't make my tools open source.
Any particular reason? The code may be hacky and not particularly pretty, but there is no commercial value to you in keeping it closed source and it gives others a chance to continue it if you drop out of the scene. Take the vqa encoder Ugordan made for example, no one can improve or port that since the source was never released for any version of it.
Also, if you have started using code from OpenRA or XCC, then barring permission from whoever wrote it, I believe the code is all under the GPL version 3 so you would be obliged to release the source for anything you wrote based on it if you wanted to comply with the license terms.
Basically there is little to no benefit to you in keeping it closed source and you may be obliged to do it anyway by other code you've incorporated. Just throw it up on github and call it done. QUICK_EDIT
i can only compile as target platform "any CPU" on my 32 bit PC.
download again and test version 1.1.0.1 if that works better.
Nice, it runs as 64-bit app in x64 system and also runs in x86 system. The
loading images is a bit faster.
Was able to make SHP out of 65534 frames and viewed it in SHP Builder.
Adding to the list (load images), it is still not able to open all the 64K files
in one step if I load the palette first and then load images. First it loaded
32760 then added again twice to get the rest.
@Blade
attached the source for the SHP reading and creating routines that i used.
Copied and converted together from OpenRA and SHP Builder.
The classes encapsulated and modified for easier usage in this project.
It's basically only
CShaPer.CreateSHP(...)//bool keepEvenFrameSizes is used to set "prevent TS wobble bug"
CShaPer.GetFrames() //reads the header only, not the entire SHP image data (the OpenRA methods adjusted a bit for this)
CShaPer.GetFrame() //returns the bitmap of a single frame
The other classes are too much intertwined with all my other tools, and i have no interest to extract the parts that are used on this tool only, nor to make all tools (including non C&C related projects) public.
I wasn't trying to force your hand or anything, most people won't even be interested in the code in this community and its not like the OpenRA team or even Olaf are really likely to pursue the matter and they are the only ones who would have standing to do anything about enforcing the GPL on you. As it is, if I can't rebuild the entire tool, a bit of the source isn't exactly useful.
You should be careful about incorporating open source code in your tools if you don't intend to play by the open source rules though, the GPL requires you make the entire source available for a binary you distribute that contains some GPL code, not just the bit of GPL code you happen to use.
I personally think its just the right thing to do to open source tools for modding. I am aware that I have released tools myself without source because they were based on code I couldn't just okay the release of on my own, but if I could have I would. QUICK_EDIT
After clicking start and the operation finishes, the cursor jumps to the status panel, which means that you can't immediately press delete to remove all frames and this slows down your work pace when you have a multiple files to process.
Also, for some reason I can't import the attached SHPs into the tool (others worked fine).
The OpenRA function to determine if they are a valid SHP returns false.
mwavemsam has 0 as compression set on all frames, valid are only 1 and 3 for SHP(TS)
same is true for ltnkcrus
Whatever program was used to create these, that program sucks at writing correct SHP files.
I'll see if i can adjust the routine to verify SHP format files to allow such bad written files.
\Edit
try version 02
it should load these bad SHPs
Version 01.01.00.02
-focus is set back to the datagrid after SHP conversion process is finished
-SHPs with wrong compression bit set are loaded anyway
-last used settings stored/loaded from ImageShaper.ini _________________ SHP Artist of Twisted Insurrection: Nod buildings
I believe I just used OS SHP Builder to create those SHPs, but there's a (small) chance that I used XCC Mixer.
Edit: apparently ImageShaper crashes whenever it's minimized by whatever means and it'll be invisible when you start it back up afterwards because it's apparently way off-screen. For some reason ImageShaper.ini then has this:
It sets the same flag that I mentioned earlier. Should benefit x86 users.
When a palette is loaded and then loading images, it assigns the palette
to the loading images which was hitting the 4gb limit, that was the reason
why it was not loading all 64k images at once. It doesn't increase memory
consumption on loading images when no palette is loaded and the program
is maximized, so loads all 64k images in one go.
On right click, reset palette or clear palette would be helpful. Also on the
left panel right click, clear all images option could be added.
Possibly a check or warning could be added if some image has not yet
set with a palette before running the start process.
A select all for one column option would be helpful if already some hotkey
is not available. Ctrl+A selects all files on all columns.
After resizing and probably when in background it crashes and sets INI values as
X_StartPosition=-32000
Y_StartPosition=-32000
Then INI needs to be deleted or those values set to 0. QUICK_EDIT
Version 01.01.00.03
-(bugfix) bad ini values for position and size don't crash anymore
-(update) simple palette manager added, which keeps track of different palette settings
-(update) image files only point to a palette from the manager and don't keep a full copy of it, thus hopefully reducing memory usage
-(update) palette edits are instantly applied to all files sharing the currently selected palette
-(update) performance of deleting files/frames improved
@E1: i've deliberately not yet set /LARGEADDRESSAWARE, due to the changed behaviour with the new palette manager.
With this, each added file should in theory only have a pointer to a very small list of different palette setups, with files/frames having the same palette setup also pointing to the same palette object. Thus the memory used for each each file should be very small and not a full copy of an entire palette setup for each frame.
Would be nice if you could test this and verify if true.
With the new palette manager, you also don't have to do "Set Palette" to the images/frames after each color modification, since the palette you see is the same object that the image/frame is using.
This should also speed up the workflow to set up the different files. _________________ SHP Artist of Twisted Insurrection: Nod buildings
You could always apply the >2gb patch, it only makes a program capable
of using 3gb on x86.
The case of loading images with a palette loaded is still the same. It reaches
the 4gb limit and doesn't load all files in one go. But the improvement is
now we can unload the palette and then load images and it loads all images
without needing the program to be maximized.
The palette manager dropdown should show full path of the palette as a
same named palette could have been loaded from some other folder.
Select all in a column works fine which replaces earlier sort function.
If you want to increase complexity, it could be made shadow frames aware
like SHP Builder. QUICK_EDIT
Ok, i'll see first if i can improve that in a more elegant way. Making it largeaddressaware seems to be a pretty cheap solution imo.
E1 Elite wrote:
If you want to increase complexity, it could be made shadow frames aware
like SHP Builder.
hmm, a simple "take second half as shadow" seems very imprecise imo (e.g. animations don't have them).
The user also has to give the shadow frames a different palette setup anyway and i have no idea how to make the right palette setup automatically, since each Modder may use different background and shadow colors when rendering stuff.
Were there any tests done showing that shadow frames need a different/certain compression to work fine?
Otherwise there is also no need to differentiate normal from shadow imo.
I have my C&C tools in a rather long subfolder
d:\Modding\Tiberian Sun\ImageShaper\
which would make the name of a palette very long and in most cases the filename itself unreadable in the short combobox. Thus i left only the filename + numbered suffix for different versions.
I wouldn't like to make the palette combobox super wide just for the path, which is usually the subfolder \Palettes _________________ SHP Artist of Twisted Insurrection: Nod buildings
Most would be using the same palette for both normal and shadow frames.
Only in case like RA2 civ buildings where a different palette is used for rubble
frames, you could automatically assign the same palette for its shadow frame.
Not sure what tests were done but SHP Builder has an option for having
best compression or RLE_0 for normal frames and none for shadow frames.
.NET has no PCX support and i don't add the huge ImageMagick library for this outdated format again.
new version uploaded
please test again
Version 01.01.00.04
-(update) images/frames store only an int index to the used palette from the palettemanager, thus reducing memory usage per image/frame
\Edit
Version 01.01.00.05
-(bugfix) palettemanager crashed on images with unassigned palettes _________________ SHP Artist of Twisted Insurrection: Nod buildings
Loading images with palette loaded case remains the same reaching 4gb limit.
Could try it in 2 passes, 1st as if no palette is loaded and 2nd assign the loaded
palette. In the meantime could show busy cursor or not show the list or show
progress bar etc. so that user knows when it has finished loading. QUICK_EDIT
No clue how palette or no palette can still make a difference.
Each cell in the DataGrid stores only a single integer now for the palette. _________________ SHP Artist of Twisted Insurrection: Nod buildings
Not sure what tests were done but SHP Builder has an option for having
best compression or RLE_0 for normal frames and none for shadow frames.
What about pcx file format support?
Its RLE, RLE compresses same bytes to one byte and repeat count, "aaaabbcdeeeeefghhhij" for example would become "a4b2c1d1e5f1g1h3i1j1".
So obviously its best for all frames, back then it would have been a exception for large ones.
however WW's only compresses 0x0 entries meaning
0 - The transparent index
* - some other index
0 0 0 0 0
0 * * * 0
0 * * * 0
0 0 0 0 0
With WW's RLE compresses as
0 [5] * * * 0 [1]
* * * 0 [1]
* * * 0 [5]
[] - zeroes to insert when decompressing
There is no other compression than RLE, the Flags entry is a bitfield,
dunno what bit 1 is, bit 2 is IsCompressed, and rest i don't know.
you check if a bit is true like Flags & 2 which tells you IsCompressed.
Careful tho, apparently some old ShapeSet version WW had stored the Flags as a word and wrote it as a dword, result of that is like in harvestr.shp, where its 0xCCCC0003. So of all flags only the second word entries are valid.
Afict the other flags aren't checked in the game but the code is big mess so i can't be sure
As for PCX, its a vary simple format, you can find standalone implementations for C# too im sure, most complex thing in it would be compression with is well well well, RLE but its more or less RLE i described at first not just zeroes. _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
yeah, at the start of this project i checked the PCX format. I only found a rough description on wikipedia and then the only free .NET library via ImageMagick.
Though i guess now with Nyerguds RLE routine and some time i could get PCX format done. However, apart from XCC Mixer which supports this as a secondary extraction format (PNG is also supported), i don't see why PCX would be necessary.
Especially since free tools like Irfanview can convert PNG to PCX and vice versa in a batch process, if someone really has old PCX lying around.
While researching a bit about SHP, i've read this topic where you among others discussed the format. Very interesting read btw.
Are there any new things found out?
Do i have to add an option for these strange Blitter bit flags? Any more info how or if they work?
Currently i write 01 when uncompressed and 03 when RLE_Zero compressed. From what i've read it should be actually 00 and 02.
However nearly all TS SHPs use 01 and 03.
I'm confused what is correct now. _________________ SHP Artist of Twisted Insurrection: Nod buildings
As I understand it, we have no idea what the first bit actually does, though as you say its is often set.
Although it takes up 4 bytes in the file, we'll assume only the first one matters (because for our purposes it is the only thing that does). For example in binary these are the 4 values you can see:
If we treat each bit as a separate boolean value, then 3 is what you get when both bools are set to true. We know the second bool indicates compression, so both 2 and 3 are valid indicators of that since they both have the second bit set. We don't know what the first one does (if anything) in the final game, but its set when the value is either 1 or 3. So basically 0 or 1 for uncompressed are valid and 2 and 3 for compressed are valid. QUICK_EDIT
Not much besides what i mention there.
Fun trivia on WW's RLE, is we found the RLE encoder in TS, its used to pack voxel shadows.
The game generates RLE compressed shadows for most if not all facings on the first time a voxel unit appears and makes a turn then the game caches them, then it only generates compressed shadows on slopes, not sure if those are cached as well, didn't seem to be, rle was called every time afict.
So yea bitfields, how you check them the crude way is bitfield & value, like
bitfield & 2 tells you if its compressed,
bitfield & 1 tells you if flag 1 is set whatever it is,
bitfield & 32 tells you if flag 6 is set
as for a byte bitfield its
bit | number value
1 | 1
2 | 2
3 | 4
4 | 8
5 | 16
6 | 32
7 | 64
8 | 128 _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
Version 01.01.00.06 uploaded
-a few adjustments on the class for the internal stored images/frames
-progressbar is showing the current progress of the imported files
Feel free to test again E1. I just successfully drag&dropped 65535 files (my public carrier frames copied several times) in one go into the tool.
Tool went up to 250MB memory usage max for all these in the datagrid.
Then pressed convert, memory usage went up to 4.2 Gigabyte (tested on a 64bit machine)
SHP created, size 1.27 GB (1.364.399.848 Bytes)
Duration: 850,0sec
Please no one ask me to upload and make this SHP public
Nice, now it loads all 64k files in one go even when palette is loaded.
Never took more than 2.4 gb for the whole test which I was testing with.
Could mention that SHP files also could be dragged and dropped. Either
Load Images could be named as Load Images/SHP or could mention in
the readme info. QUICK_EDIT
So you got
namespace ImageShaper
{
public enum SHP_TS_EncodingFormat
{
Undefined,
Uncompressed,
RLE_Zero,
Detect_best_size,
Unknown0,
Unknown2
}
}
switch (array[i].Format)
{
case 0:
cimageFile.CompressionFormat = SHP_TS_EncodingFormat.Unknown0;
break;
case 1:
cimageFile.CompressionFormat = SHP_TS_EncodingFormat.Uncompressed;
break;
case 2:
cimageFile.CompressionFormat = SHP_TS_EncodingFormat.Unknown2;
break;
case 3:
cimageFile.CompressionFormat = SHP_TS_EncodingFormat.RLE_Zero;
break;
default:
cimageFile.CompressionFormat = SHP_TS_EncodingFormat.Undefined;
break;
}
----------------------------------------------------
No just no...
So
//assuming its a byte bitfield we get
enum Shape_TS_Frame_Flags
{
UnknownBit1 = 1,
IsCompressed = 2,
UnknownBit3 = 4,
UnknownBit4 = 8,
UnknownBit5 = 16,
UnknownBit6 = 32,
UnknownBit7 = 64,
UnknownBit8 = 128,
}
these are defined like this so you don't have a cryptic bitfield & 128
//this is wrong
if(array[i].Format & (UnknownBit1 | UnknownBit3) // this is what a number 5 is, bit 1 and 3;
{
cimageFile.CompressionFormat = SHP_TS_EncodingFormat.Unknown2;
}
now in your frame processing code you use those bools in SHP_Frame_Bools,
i'd turn CompressionFormat in to a bitfield and check the bits as processing but whatever
-----------------------------------------------------
now to set a bitfield for when writing the shape frame
//setting bit 1
FrameFlags =| UnknownBit1;
//Setting bit 1 and 2
FrameFlags =| (UnknownBit1 | FrameCompressed); _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
yes, i have to overhaul this entire bit flag part as this is pretty incompatible with the current way of handling the possible ways to write a frame (which were only 2 when i started this project).
Though not sure if worth the work, if it's only these 2 ways in the first place and all these unknown bits might be just useless.
But for sake of completeness i'll add the bit flag mechanism anyway and also add a way to allow the user to set/read them. Then maybe only in a "pro mode" so the normal user in "simple mode" doesn't have to care about these strange flags and only has the 3 options, uncompressed, RLE_Zero, detect_best_size. _________________ SHP Artist of Twisted Insurrection: Nod buildings
They may be useless they may be not, but they are there, checking as Flags & 2 ensures you know if a shp is rle compressed for all 3 known to be used numbers which are 2, 3 and 5.
One other note in Frame header after what you call RadarColor there's a dword what XCC called Zeros, its not always zero as some would claim, RA2's money.shp and tstlexp.shp have 00FB1200, no clue whats going on there. _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite Last edited by tomsons26lv on Fri Dec 15, 2017 1:26 am; edited 1 time in total QUICK_EDIT
is it 3 byte radarcolor or 4 (with alpha?) and then followed by the dword?
also i've read something about the very first byte in a SHP not always being 00. Any more insight in that? Do i have to take care of it? _________________ SHP Artist of Twisted Insurrection: Nod buildings
What your calling radar colour is not, its the average colour of the SHP file if it was to be viewed at a small size, what happens to be also used for the radar.
TS/RA2 does not use RGBA, only RGB, alpha is handled directly by a specifically blitter that takes the shape and blits it with a fixed transparency (25, 50, 75). QUICK_EDIT
You can post new topics in this forum You can reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
Image Shaper
)
