The command line for the server setup is basically like this
- /INSTALLDIR="C:\Program Files\Microsoft Application Virtualization Server"
- /MANAGEMENT_WEBSITE_NAME="Microsoft App-V Management Service"
While trying to install nothing happened at first so I noticed that the problem was somehow related to /INSTALLDIR, while using install dir I got "appv_server_setup.exe" looping in task manager and had about 300 instances running and the computer was crazy slow.
I tried to disable a lot of things such as uac, created the directory and set the security to everyone full control, ran the command line as administrator, tried to change the directory path on the install dir but no avail.
While trying this I checked logs witch are located under "c:\Users\%username\appdata\Local\Temp" the setup creates a log there. Even thou I tried to change the install dir to some other location the install had forced the install to install on the "[ProgramFiles64Folder]\Microsoft Application Virtualization Server". I tried to skip the "\INSTALLDIR=" and got an error in the log saying the parameters are wrong as you can see
[03C8:0B64][2012-07-04T21:53:53]: AppvUX: Unexpected command line options supplied. Make sure that you've selected the feature for which you've provided configuration options.
[03C8:0B64][2012-07-04T21:53:53]: AppvUX: Command line options are invalid. Please make sure that it follows the following format, /<ParameterName>=<Value> /<ParameterName>="<Value>" /<Parameter3>
What ever I did even tried to debug the issue with Sysinternals Process Monitor I did not find the solution for this problem, so I realized this is only a BETA so it must be some bug.
Note, the word goes around that .NET 4.0 should be installed, but in this instance .NET 4.0 and older are installed.
PS. Thanks Stefan for this question :)