Keygenning Webservices

[surferxyz]

My general flow for reversing something like this follows these type of steps:

Figure out how to gain access to the VM while it is running for analysis (eg. if the VM guest is linux without disk encryption, than a possibility would be to add a SSH authorized_keys file directly to the disk, then reboot the VM ...).

Now that you have access to the running environment, it is a lot easier to figure out what the "serial accepting" application is. (eg. you can look at the process listing, or see what application is listening on a particular port, or view configuration files...)

In your case you made it sound like it is part of a web application, so now you just need to figure out where that web application is, what language it is written in etc, and reverse engineer it...

You can usually be super lazy and just search the entire disk for the name of one of the web application files (eg. maybe when visiting the appliance you go to /index.php, you can just search the entire disk for index.php and probably you find it.).

If not figure out what the webserver that is handling your request is (you mentioned tomcat), than either look at the configs for that, or be lazy and search the whole disk for associated server files (eg. search the whole disk for .war)

Other than the kinda annoyance to drill down into the VM to find what to analyze, there is no other real differences than regular reverse engineering... although I guess you could say "figuring out what the appliance does" is just regular reverse engineering.

You specifically ask "how to intercept the api calls", well with access to the VM you can do it the same way as you normally would depending on the OS... It kinda sounds like you are confused because you are not familiar with reverse engineering outside of a windows environment... I can give you one good tip, use the IDA remote debug server executables, you can drop the linux server binary into the VM run it, then use the same already familiar IDA debugging environment on windows to debug your target inside the VM

I think you will need to be more specific about what you are trying to reverse, I am guessing a java web application, you can reverse those the same way you would any other java code.