99 hf qwen instruct

Qwen Instruct¶

In [3]:

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline
from langchain_core.prompts import ChatPromptTemplate
from langchain_huggingface import HuggingFacePipeline
import torch

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline from langchain_core.prompts import ChatPromptTemplate from langchain_huggingface import HuggingFacePipeline import torch

In [4]:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen3-4B-Instruct-2507"

## Quantinize model
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    quantization_config=bnb_config,
    device_map="auto"
)

from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "Qwen/Qwen3-4B-Instruct-2507" ## Quantinize model bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16 ) # load the tokenizer and the model tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", quantization_config=bnb_config, device_map="auto" )

Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]

In [10]:

# prepare the model input
prompt = """
<Dbtr>
<Nm>Nardi sunardi</Nm>
<PstlAdr>
<AdrLine>Buliding Sudan State, near park emerelad</AdrLine>
<AdrLine>Street South china,United States</AdrLine>
</PstlAdr>
</Dbtr>

Dari mana negara asal debtor ?"""
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.7,
    top_p=0.8,
    top_k=20
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 

content = tokenizer.decode(output_ids, skip_special_tokens=True)

print("content:", content)

# prepare the model input prompt = """ Nardi sunardi Buliding Sudan State, near park emerelad Street South china,United States Dari mana negara asal debtor ?""" messages = [ {"role": "user", "content": prompt} ] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device) # conduct text completion generated_ids = model.generate( **model_inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.8, top_k=20 ) output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() content = tokenizer.decode(output_ids, skip_special_tokens=True) print("content:", content)

content: Dari data yang diberikan:

```
<AdrLine>Street South china,United States</AdrLine>
```

Ada kesalahan dalam penulisan alamat. Kata **"South china"** tidak merupakan nama negara yang sah, dan **"United States"** adalah nama negara yang benar.

Namun, dalam konteks ini, penulisan **"Street South china, United States"** sangat mungkin merupakan kesalahan ketik atau kesalahan terjemahan. Karena **"South China"** bukan nama negara, dan **"United States"** adalah negara di Amerika Serikat.

Jadi, meskipun alamatnya ditulis "near park emerelad" dan "Street South china, United States", maka **negara asal debtor** yang dimaksud adalah:

👉 **United States**

**Kesimpulan:**  
Negara asal debtor adalah **United States**.  

*(Catatan: "South China" bukan negara, dan tidak dapat menjadi negara asal. Jika maksudnya "South China" sebagai wilayah, maka itu bukan negara, dan negara asal tetap di United

Its still low, , i want compered using Vllm¶

marih liat

In [ ]: