Running DeepSeek R1 locally
 

DeepSeek has its flagship V3 model equivalent to GPT4 and it's reasoning model R1 freely accessible: AI training for 5.6 million USD exceeding the quality of 100 mil to 1 bil USD. Inference of high quality is within reach of your own local environment where your data stays private. I've found their models better at reasoning than OpenAIs significantly. It's quite exciting and I'm surprised noone has brought the topic up yet, given the large amount of use cases for reverse engineering and the very low cost.

If you have a GPU recommend OLlama which works on Windows, Linux and Mac (can also rin Facebook/Meta's Llama models): . Can choose from the models listed here: . 8b is pretty lightweight but if you have a recent Nvidia GPU with a lot of RAM why not go for 32b.

For a frontend chat interface, I recommend Chatbot AI: Even better the V3 and R1 model are open source and you can do your own model finishing if you have the resources.

R1: V3:

Are there any reverse engineering (especially for decompilation), specific models?

This makes DIGITS even more interesting to me, considering buying one. (But it will only be out in may). https://www.nvidia.com/en-eu/project-digits/

What I dont get though is how 1PFLOG in FP4 is a selling point when the regular GTX5090 has like 600 TFLOP in FP32? Am I missing something or is the 128GB integrated RAM the selling factor?

There is even more powerfull ML model then DeepSeek - Alibaba’s Qwen2.5-Max

DeepSeek censorship:

It would be nice to train an RE model. The good news now is that training is being shown to be feasible possibly on an academic grant level budget. Someone should train a proper open source RE model at some point.

1PFLOP FP4 is a marketing gimmick maybe, that amount of RAM is a big plus tho. The new DeepSeek models use FP8 and have shown it's reliable for training, a good breakthrough. Sounds good enough to run good size models at moderate load.

Alibaba sounds interesting haven't heard much about it.

By the way DeepSeek censorship from demos I saw is on the website but at least running R1 locally, it seems to not be censoring those things much or at all.

Censorship update: it appears if you download models and run locally there is no censorship.

Censoring is definitely done on their public website if you run your queries on their hardware. Here is how it works:
when you send your query it goes into the real model avatars thinking or generating a response.

At the same time, it is sent to a classifier model that is far cheaper, faster and specific. This classifier is trained with a prompt similar to "is the following '<prompt>' related to the following list of sensitive topics". If it returns yes, the main query is immediately aborted and a message displayed. Sometimes you can see it start to think and cut off a few sentences into it's deepthink. Other time it cuts off so fast that it appears nothing yet was emitted.

No problem, run it locally, you won't deal with the censorship classifier. It would be interesting though to have a list of the sensitive topics but that is probably kept secretly and securely.

Update: According to this article I am mistaken and the censorship concerns mentioned are legit:

I only can prompt twice, then deepseek says,

I asked it about Falon Gong last night (locally) - and spent about 15 minutes debating it.

But let's be real.. ? Does anyone except it to behave otherwise--if so, you're kidding yourself and need to stop.

Locally its quite a useful tool... especially if you are playing with some of the more 'schizo' rethinking builds available via huggingface.

Regards,

Fyyre

I know its a late reply but the issue is still present. The reason is the recent spike in popularity through tons of youtube videos, tik toks etc etc. DeepSeek is under heavy pressure and currently not able to handle the user load. Their API was down for roughly 10 days at the beginning of february aswell.

Besides the normal user load, they currently have to manage, they are currently still under heavy DDOS attacks. Because of that its currently not even possible to top up API Credits :(

Hopefully they will resolve this soon.

In the meantime if you have an X Account Grok 3 seems to be the new kid on the block and atleast yesterday I was ablo to use it without any subscriptions.

Dont get used to it tho they hiked up the prices (they doubled them) and probably will get their asses kicked for it since nobody I know will pay 40$ a month for access to a Chat LLM if you can have slightly worse for 1/10 of the price.

Best regards

Hi Guys,

there is a nice publication about this here: https://arxiv.org/pdf/2505.19915

Does anyone here know more about this topic and used such agents?


Best regards

Is there any tutorial on how to fine-tune Deepseek Coder V2?

I am facing dependency hell when trying to setup an environment that can be used for additional training on private C source code.

ollama is good for running it on Windows/WSL, but not for additional training.

Regards,
Tom

This one is new too

https://huggingface.co/fdtn-ai/Foundation-Sec-8B

There are discussions on the Hugging Face page (e.g., fdtn-ai/Foundation-Sec-8B/discussions/10) about running this model with Ollama. Users have attempted it, and there's an Ollama entry for an "abliterated" (uncensored) version huihui_ai/foundation-sec-abliterated:8b

By the way note the name: abliterated … which is a clash of two words obliterated and ablated. This because they depotentiated some neurons responsible for censorship. It’s a full AI hack at its best. This is the future.

Somehow you can imagine it as a sort of neural surgery

Also.

1. Find open Ollama servers, using for example shodan command line or the following script Ollama Hunter (which by the way you should update to the new shodan APIs — don’t be dumb, just ask an LLM to do it for you)
2. Alternatively use shodan — shodan count port:11434 product:"Ollama" country:XX where XX is your country code
3. In this case the shodan command line is — https://help.shodan.io/command-line-interface/0-installation
4. Whatever way you follow you need a shodan key, or the polito cookie. You can find one with a bit of Google dork — es. look here https://www.exploit-db.com/google-hacking-database
5. Connects to the Ollama that are open via chatbot AI or similar apps and of course using a VPN and you’re done
6. Use bigger models, if you want/dare


Ollama Hunter is a Python tool that searches Shodan for publicly exposed Ollama LLM instances running on port 11434, and retrieves the list of available models from each host.
This tool is designed for researchers, security analysts, and red teamers who want to map the exposure of open LLM endpoints on the internet.

https://github.com/saadi1995/ollama-hunter

Isn't this basically stealing from innocent users who do not have enough knowledge to secure their Ollama instances?
Inferences which use GPU are not cheap and can run the bill up to thousands of dollars in some cases. That too, you are suggesting the use of bigger models (more expensive for the user)! :eek:

Downloading and using pirated software does not directly harm the end user but stealing paid compute of innocent people in this way is plain unethical and nothing but stealing... :confused:

First, I said how it can be done, not that you must do it. That's up to you

"downloading and using pirated sw does not harm the end user" It's arguable, because it harms the developer.

In general, this is the wrong place for out-of-scope considerations, you're on exetools, not on reddit' innocent-souls channel.

at that point also the existence of that Ollama Hunter would be considered that way ... so leave these considerations for X.

That's quite rude of you.
Someone did it to me and it resulted in a bill of several hundred dollars. It's extremely painful when this happens.

I wasn't even leaving the server unattended; I was actively configuring it when someone decided to scan and leech it.
Detailing such methods only serves to make it easier for inexperienced users like me to become targets for attacks!

Exetools was never a place where methods to take advantage of innocent online users were posted! There are also no posts to teach server hacking on Exetools.

the conclusion is - do not leave plain endpoints to the public...
think about auth

The lesson would be do not be dumb

It is not good to call anyone dumb. :( You may be intelligent no doubt, but it's very bad to call anyone as dumb.

You've effectively posted instructions to drain funds from someone's credit card without consent by hacking their servers.
Not much different from using stolen credit cards.

I hope the quality of the posts in the Exetools forum do not degrade to those in the (now) taken-down CRACKED.TO forum.
There is a very good reason why the CRACKED.TO forum was taken down by the LEA: Because they dealt with stolen user accounts in one form or the other.

I see that you've set up a new merchant site. It's nice, by the way :)

+1 .. open ports on the Internet = bad .. nothing new

Shub Nigurath should indeed make it very clear that the consequences of draining money from another person's credit card in this way is equal to using a stolen credit card, leading to serious jail time. Most users here would not be okay with using stolen credit cards in any way.

Most of the consumer-grade VPNs do not offer enough anonymity against financial crimes (which is good).
The warning should be very clear that if someone starts to drain in an unauthorized way credit cards linked to unsecured servers by using their expensive GPU compute resources for inference, they could go to jail.

While some users might accept illegal activities, the majority on the Exetools forum would strongly oppose financial crimes of this scale that could lead to imprisonment. Therefore, the warning needs to be very clear.

Also...
-1: It is never okay to call anyone dumb.

This is all I want to say, in this post and the one above.

So as far as I know, if someone puts an instance publicly on the web it is free to use. This is not hacking at all as there is no unauthorized intrusion is no authorization is needed. If using default credentials, this is at least arguably not an authorization mechanism though it touches a gray area. This is not causing damage or harm to the server either but using it as intended. I do think responsibility for guarding resources lies on the hoster of such services.

Also I dont think he called anyone dumb, he described the action of making a costly service unsecured as dumb. Ive never liked labels or names applied to people. But actions I feel are totally fair to criticize. We all tend to make stupid mistakes in life, but that doesnt say anything about us after we learn from them. Dumb when describing a person means inability to speak, which is very different than when describing an action.

So im unclear on the legality and ethically here. Likely if passwordless then it is legal. If its default password, less likely legal but possibly. If the password had to be obtained through unauthorized efforts whether brute force, or online leaks, etc it is not legal. Ethically if you know you are running large bills then its not ethical. Unless the poster gave explicit consent like a public notice it is free to use or they privately agreed, etc. Then again, everyone probably knows all of this already.