[RFC v2 0/2] target/loongarch: Add loongson binary translation feature

[RFC v2 0/2] target/loongarch: Add loongson binary translation feature

[Bibo Mao]

Loongson Binary Translation (LBT) is used to accelerate binary
translation. LBT feature is added in kvm mode, not supported in TCG
mode since it is not emulated. And only LBT feature is added here, LBT
registers saving and restoring is not supported since it depeeds on LBT
feautre implemented in KVM kernel

---
v1 ... v2:
  1. Add LBT register saving and restoring in vmstate
  2. Add two pseudo feature flags: default_features and forced_features.
---

Bibo Mao (2):
  target/loongarch: Add loongson binary translation feature
  target/loongarch: Implement lbt registers save/restore function

 target/loongarch/cpu.c                | 69 ++++++++++++++++++++++++
 target/loongarch/cpu.h                | 24 +++++++++
 target/loongarch/kvm/kvm.c            | 78 +++++++++++++++++++++++++++
 target/loongarch/kvm/kvm_loongarch.h  | 16 ++++++
 target/loongarch/loongarch-qmp-cmds.c |  2 +-
 target/loongarch/machine.c            | 24 +++++++++
 6 files changed, 212 insertions(+), 1 deletion(-)


base-commit: ffdd099a782556b9ead26551a6f1d070a595306d
-- 
2.39.3

[RFC v2 1/2] target/loongarch: Add loongson binary translation feature

[Bibo Mao]

Loongson Binary Translation (LBT) is used to accelerate binary
translation, which contains 4 scratch registers (scr0 to scr3), x86/ARM
eflags (eflags) and x87 fpu stack pointer (ftop).

Now LBT feature is added in kvm mode, not supported in TCG mode since
it is not emulated. There are two feature flags such as forced_features
and default_features for each vcpu, the real feature is still in cpucfg.
Flag forced_features is parsed from command line, default_features is
parsed from cpu type.

Flag forced_features has higher priority than flag default_features,
default_features will be used if there is no command line option for LBT
feature. If the feature is not supported with KVM host, it reports error
and exits if forced_features is set, else it disables feature and continues
if default_features is set.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 target/loongarch/cpu.c                | 69 +++++++++++++++++++++++++++
 target/loongarch/cpu.h                | 12 +++++
 target/loongarch/kvm/kvm.c            | 26 ++++++++++
 target/loongarch/kvm/kvm_loongarch.h  | 16 +++++++
 target/loongarch/loongarch-qmp-cmds.c |  2 +-
 5 files changed, 124 insertions(+), 1 deletion(-)

diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index b5c1ec94af..d9d601da07 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -380,6 +380,8 @@ static void loongarch_la464_initfn(Object *obj)
     CPULoongArchState *env = &cpu->env;
     int i;
 
+    env->default_features = 0;
+    env->forced_features  = 0;
     for (i = 0; i < 21; i++) {
         env->cpucfg[i] = 0x0;
     }
@@ -413,6 +415,7 @@ static void loongarch_la464_initfn(Object *obj)
     data = FIELD_DP32(data, CPUCFG2, LSPW, 1);
     data = FIELD_DP32(data, CPUCFG2, LAM, 1);
     env->cpucfg[2] = data;
+    env->default_features |= BIT_ULL(LOONGARCH_FEATURE_LBT);
 
     env->cpucfg[4] = 100 * 1000 * 1000; /* Crystal frequency */
 
@@ -571,6 +574,35 @@ static void loongarch_cpu_disas_set_info(CPUState *s, disassemble_info *info)
     info->print_insn = print_insn_loongarch;
 }
 
+static void loongarch_cpu_check_lbt(CPUState *cs, Error **errp)
+{
+    CPULoongArchState *env = cpu_env(cs);
+    enum loongarch_features feature;
+    bool kvm_supported;
+
+    feature = LOONGARCH_FEATURE_LBT;
+    kvm_supported = kvm_feature_supported(cs, feature);
+    if (env->forced_features & BIT_ULL(feature)) {
+        if (kvm_supported) {
+            env->cpucfg[2] = FIELD_DP32(env->cpucfg[2], CPUCFG2, LBT_ALL, 7);
+        } else {
+            error_setg(errp, "'lbt' feature not supported by KVM on this host");
+            return;
+        }
+    } else if (env->default_features & BIT_ULL(feature)) {
+        if (kvm_supported) {
+            env->cpucfg[2] = FIELD_DP32(env->cpucfg[2], CPUCFG2, LBT_ALL, 7);
+        }
+    }
+}
+
+static void loongarch_cpu_feature_realize(CPUState *cs, Error **errp)
+{
+    if (kvm_enabled()) {
+        loongarch_cpu_check_lbt(cs, errp);
+    }
+}
+
 static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)
 {
     CPUState *cs = CPU(dev);
@@ -584,6 +616,11 @@ static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)
     }
 
     loongarch_cpu_register_gdb_regs_for_features(cs);
+    loongarch_cpu_feature_realize(cs, &local_err);
+    if (local_err != NULL) {
+        error_propagate(errp, local_err);
+        return;
+    }
 
     cpu_reset(cs);
     qemu_init_vcpu(cs);
@@ -643,12 +680,44 @@ static void loongarch_set_lasx(Object *obj, bool value, Error **errp)
     }
 }
 
+static bool loongarch_get_lbt(Object *obj, Error **errp)
+ {
+    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
+    bool ret;
+
+    ret = false;
+    /* lbt is enabled only in kvm mode, not supported in tcg mode */
+    if (cpu->env.forced_features & BIT_ULL(LOONGARCH_FEATURE_LBT)) {
+        ret = true;
+     }
+    return ret;
+}
+
+static void loongarch_set_lbt(Object *obj, bool value, Error **errp)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
+
+    if (!kvm_enabled()) {
+        return;
+    }
+
+    if (value) {
+        /* Enable binary translation for all architectures */
+        cpu->env.forced_features |= BIT_ULL(LOONGARCH_FEATURE_LBT);
+    } else {
+        /* Disable default features also */
+        cpu->env.default_features &= ~BIT_ULL(LOONGARCH_FEATURE_LBT);
+    }
+}
+
 void loongarch_cpu_post_init(Object *obj)
 {
     object_property_add_bool(obj, "lsx", loongarch_get_lsx,
                              loongarch_set_lsx);
     object_property_add_bool(obj, "lasx", loongarch_get_lasx,
                              loongarch_set_lasx);
+    object_property_add_bool(obj, "lbt", loongarch_get_lbt,
+                             loongarch_set_lbt);
 }
 
 static void loongarch_cpu_init(Object *obj)
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index 41b8e6d96d..36fb160a8c 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -152,6 +152,7 @@ FIELD(CPUCFG2, LLFTP_VER, 15, 3)
 FIELD(CPUCFG2, LBT_X86, 18, 1)
 FIELD(CPUCFG2, LBT_ARM, 19, 1)
 FIELD(CPUCFG2, LBT_MIPS, 20, 1)
+FIELD(CPUCFG2, LBT_ALL, 18, 3)
 FIELD(CPUCFG2, LSPW, 21, 1)
 FIELD(CPUCFG2, LAM, 22, 1)
 
@@ -280,6 +281,10 @@ struct LoongArchTLB {
 typedef struct LoongArchTLB LoongArchTLB;
 #endif
 
+enum loongarch_features {
+    LOONGARCH_FEATURE_LBT, /* loongson binary translation extension */
+};
+
 typedef struct CPUArchState {
     uint64_t gpr[32];
     uint64_t pc;
@@ -289,6 +294,13 @@ typedef struct CPUArchState {
     uint32_t fcsr0;
 
     uint32_t cpucfg[21];
+    /*
+     * Features not specified from command line
+     * Comes from cpu type and kvm host capability
+     */
+    uint64_t default_features;
+    /* features parsed from command line, such as lbt=on */
+    uint64_t forced_features;
 
     /* LoongArch CSRs */
     uint64_t CSR_CRMD;
diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
index 8e6e27c8bf..55e85eff15 100644
--- a/target/loongarch/kvm/kvm.c
+++ b/target/loongarch/kvm/kvm.c
@@ -780,6 +780,32 @@ int kvm_loongarch_set_interrupt(LoongArchCPU *cpu, int irq, int level)
     return kvm_vcpu_ioctl(cs, KVM_INTERRUPT, &intr);
 }
 
+bool kvm_feature_supported(CPUState *cs, enum loongarch_features feature)
+{
+    int ret;
+    struct kvm_device_attr attr;
+
+    switch (feature) {
+    case LOONGARCH_FEATURE_LBT:
+        /*
+         * Return all if all the LBT features are supported such as:
+         *  KVM_LOONGARCH_VM_FEAT_X86BT
+         *  KVM_LOONGARCH_VM_FEAT_ARMBT
+         *  KVM_LOONGARCH_VM_FEAT_MIPSBT
+         */
+        attr.group = KVM_LOONGARCH_VM_FEAT_CTRL;
+        attr.attr = KVM_LOONGARCH_VM_FEAT_X86BT;
+        ret = kvm_vm_ioctl(kvm_state, KVM_HAS_DEVICE_ATTR, &attr);
+        attr.attr = KVM_LOONGARCH_VM_FEAT_ARMBT;
+        ret |= kvm_vm_ioctl(kvm_state, KVM_HAS_DEVICE_ATTR, &attr);
+        attr.attr = KVM_LOONGARCH_VM_FEAT_MIPSBT;
+        ret |= kvm_vm_ioctl(kvm_state, KVM_HAS_DEVICE_ATTR, &attr);
+        return (ret == 0);
+    default:
+        return false;
+    }
+}
+
 void kvm_arch_accel_class_init(ObjectClass *oc)
 {
 }
diff --git a/target/loongarch/kvm/kvm_loongarch.h b/target/loongarch/kvm/kvm_loongarch.h
index d945b6bb82..bdb4f180eb 100644
--- a/target/loongarch/kvm/kvm_loongarch.h
+++ b/target/loongarch/kvm/kvm_loongarch.h
@@ -13,4 +13,20 @@
 int  kvm_loongarch_set_interrupt(LoongArchCPU *cpu, int irq, int level);
 void kvm_arch_reset_vcpu(CPULoongArchState *env);
 
+#ifdef CONFIG_KVM
+/*
+ * kvm_feature_supported:
+ *
+ * Returns: true if KVM supports specified feature
+ * and false otherwise.
+ */
+bool kvm_feature_supported(CPUState *cs, enum loongarch_features feature);
+#else
+static inline bool kvm_feature_supported(CPUState *cs,
+                                         enum loongarch_features feature)
+{
+    return false;
+}
+#endif
+
 #endif
diff --git a/target/loongarch/loongarch-qmp-cmds.c b/target/loongarch/loongarch-qmp-cmds.c
index 8721a5eb13..c6f6e1ef85 100644
--- a/target/loongarch/loongarch-qmp-cmds.c
+++ b/target/loongarch/loongarch-qmp-cmds.c
@@ -40,7 +40,7 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
 }
 
 static const char *cpu_model_advertised_features[] = {
-    "lsx", "lasx", NULL
+    "lsx", "lasx", "lbt", NULL
 };
 
 CpuModelExpansionInfo *qmp_query_cpu_model_expansion(CpuModelExpansionType type,
-- 
2.39.3

[RFC v2 2/2] target/loongarch: Implement lbt registers save/restore function

[Bibo Mao]

Six registers scr0 - scr3, eflags and ftop are added in percpu vmstate.
And two functions kvm_loongarch_get_lbt/kvm_loongarch_put_lbt are added
to save/restore lbt registers.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 target/loongarch/cpu.h     | 12 +++++++++
 target/loongarch/kvm/kvm.c | 52 ++++++++++++++++++++++++++++++++++++++
 target/loongarch/machine.c | 24 ++++++++++++++++++
 3 files changed, 88 insertions(+)

diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index 36fb160a8c..8fc99b8ee8 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -285,6 +285,17 @@ enum loongarch_features {
     LOONGARCH_FEATURE_LBT, /* loongson binary translation extension */
 };
 
+typedef struct  LoongArchBT {
+    /* scratch registers */
+    uint64_t scr0;
+    uint64_t scr1;
+    uint64_t scr2;
+    uint64_t scr3;
+    /* loongarch eflags */
+    uint64_t eflags;
+    uint64_t ftop;
+} lbt_t;
+
 typedef struct CPUArchState {
     uint64_t gpr[32];
     uint64_t pc;
@@ -292,6 +303,7 @@ typedef struct CPUArchState {
     fpr_t fpr[32];
     bool cf[8];
     uint32_t fcsr0;
+    lbt_t  lbt;
 
     uint32_t cpucfg[21];
     /*
diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
index 55e85eff15..c9c240a573 100644
--- a/target/loongarch/kvm/kvm.c
+++ b/target/loongarch/kvm/kvm.c
@@ -475,6 +475,48 @@ static int kvm_loongarch_put_regs_fp(CPUState *cs)
     return ret;
 }
 
+static int kvm_loongarch_put_lbt(CPUState *cs)
+{
+    CPULoongArchState *env = cpu_env(cs);
+    int ret;
+
+    /* check whether vm support LBT firstly */
+    if (FIELD_EX32(env->cpucfg[2], CPUCFG2, LBT_ALL) != 7) {
+        return 0;
+    }
+
+    /* set six LBT registers including scr0-scr3, eflags, ftop */
+    ret = kvm_set_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR0, &env->lbt.scr0);
+    ret |= kvm_set_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR1, &env->lbt.scr1);
+    ret |= kvm_set_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR2, &env->lbt.scr2);
+    ret |= kvm_set_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR3, &env->lbt.scr3);
+    ret |= kvm_set_one_reg(cs, KVM_REG_LOONGARCH_LBT_EFLAGS, &env->lbt.eflags);
+    ret |= kvm_set_one_reg(cs, KVM_REG_LOONGARCH_LBT_FTOP, &env->lbt.ftop);
+
+    return ret;
+}
+
+static int kvm_loongarch_get_lbt(CPUState *cs)
+{
+    CPULoongArchState *env = cpu_env(cs);
+    int ret;
+
+    /* check whether vm support LBT firstly */
+    if (FIELD_EX32(env->cpucfg[2], CPUCFG2, LBT_ALL) != 7) {
+        return 0;
+    }
+
+    /* get six LBT registers including scr0-scr3, eflags, ftop */
+    ret = kvm_get_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR0, &env->lbt.scr0);
+    ret |= kvm_get_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR1, &env->lbt.scr1);
+    ret |= kvm_get_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR2, &env->lbt.scr2);
+    ret |= kvm_get_one_reg(cs, KVM_REG_LOONGARCH_LBT_SCR3, &env->lbt.scr3);
+    ret |= kvm_get_one_reg(cs, KVM_REG_LOONGARCH_LBT_EFLAGS, &env->lbt.eflags);
+    ret |= kvm_get_one_reg(cs, KVM_REG_LOONGARCH_LBT_FTOP, &env->lbt.ftop);
+
+    return ret;
+}
+
 void kvm_arch_reset_vcpu(CPULoongArchState *env)
 {
     env->mp_state = KVM_MP_STATE_RUNNABLE;
@@ -608,6 +650,11 @@ int kvm_arch_get_registers(CPUState *cs)
         return ret;
     }
 
+    ret = kvm_loongarch_get_lbt(cs);
+    if (ret) {
+        return ret;
+    }
+
     ret = kvm_loongarch_get_mpstate(cs);
     return ret;
 }
@@ -636,6 +683,11 @@ int kvm_arch_put_registers(CPUState *cs, int level)
         return ret;
     }
 
+    ret = kvm_loongarch_put_lbt(cs);
+    if (ret) {
+        return ret;
+    }
+
     ret = kvm_loongarch_put_mpstate(cs);
     return ret;
 }
diff --git a/target/loongarch/machine.c b/target/loongarch/machine.c
index 08a7fa5370..aa2d3db601 100644
--- a/target/loongarch/machine.c
+++ b/target/loongarch/machine.c
@@ -110,6 +110,29 @@ static const VMStateDescription vmstate_lasx = {
     },
 };
 
+static bool lbt_needed(void *opaque)
+{
+    LoongArchCPU *cpu = opaque;
+
+    return !!FIELD_EX64(cpu->env.cpucfg[2], CPUCFG2, LBT_ALL);
+}
+
+static const VMStateDescription vmstate_lbt = {
+    .name = "cpu/lbt",
+    .version_id = 0,
+    .minimum_version_id = 0,
+    .needed = lbt_needed,
+    .fields = (const VMStateField[]) {
+        VMSTATE_UINT64(env.lbt.scr0,   LoongArchCPU),
+        VMSTATE_UINT64(env.lbt.scr1,   LoongArchCPU),
+        VMSTATE_UINT64(env.lbt.scr2,   LoongArchCPU),
+        VMSTATE_UINT64(env.lbt.scr3,   LoongArchCPU),
+        VMSTATE_UINT64(env.lbt.eflags, LoongArchCPU),
+        VMSTATE_UINT64(env.lbt.ftop,   LoongArchCPU),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
 #if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
 static bool tlb_needed(void *opaque)
 {
@@ -219,6 +242,7 @@ const VMStateDescription vmstate_loongarch_cpu = {
 #if defined(CONFIG_TCG) && !defined(CONFIG_USER_ONLY)
         &vmstate_tlb,
 #endif
+        &vmstate_lbt,
         NULL
     }
 };
-- 
2.39.3

Re: [RFC v2 1/2] target/loongarch: Add loongson binary translation feature

[Philippe Mathieu-Daudé]

Hi Bibo,

On 27/5/24 10:35, Bibo Mao wrote:
> Loongson Binary Translation (LBT) is used to accelerate binary
> translation, which contains 4 scratch registers (scr0 to scr3), x86/ARM
> eflags (eflags) and x87 fpu stack pointer (ftop).
> 
> Now LBT feature is added in kvm mode, not supported in TCG mode since
> it is not emulated. There are two feature flags such as forced_features
> and default_features for each vcpu, the real feature is still in cpucfg.
> Flag forced_features is parsed from command line, default_features is
> parsed from cpu type.
> 
> Flag forced_features has higher priority than flag default_features,
> default_features will be used if there is no command line option for LBT
> feature. If the feature is not supported with KVM host, it reports error
> and exits if forced_features is set, else it disables feature and continues
> if default_features is set.
> 
> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
> ---
>   target/loongarch/cpu.c                | 69 +++++++++++++++++++++++++++
>   target/loongarch/cpu.h                | 12 +++++
>   target/loongarch/kvm/kvm.c            | 26 ++++++++++
>   target/loongarch/kvm/kvm_loongarch.h  | 16 +++++++
>   target/loongarch/loongarch-qmp-cmds.c |  2 +-
>   5 files changed, 124 insertions(+), 1 deletion(-)


> +static void loongarch_set_lbt(Object *obj, bool value, Error **errp)
> +{
> +    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
> +
> +    if (!kvm_enabled()) {

Either set errp, ...

> +        return;
> +    }
> +
> +    if (value) {
> +        /* Enable binary translation for all architectures */
> +        cpu->env.forced_features |= BIT_ULL(LOONGARCH_FEATURE_LBT);
> +    } else {
> +        /* Disable default features also */
> +        cpu->env.default_features &= ~BIT_ULL(LOONGARCH_FEATURE_LBT);
> +    }
> +}
> +
>   void loongarch_cpu_post_init(Object *obj)
>   {
>       object_property_add_bool(obj, "lsx", loongarch_get_lsx,
>                                loongarch_set_lsx);
>       object_property_add_bool(obj, "lasx", loongarch_get_lasx,
>                                loongarch_set_lasx);

... or only add the property if KVM is enabled:

    if (kvm_enabled()) {

> +    object_property_add_bool(obj, "lbt", loongarch_get_lbt,
> +                             loongarch_set_lbt);
>   }

Re: [RFC v2 0/2] target/loongarch: Add loongson binary translation feature

[Philippe Mathieu-Daudé]

Hi Bibo,

On 27/5/24 10:34, Bibo Mao wrote:
> Loongson Binary Translation (LBT) is used to accelerate binary
> translation. LBT feature is added in kvm mode, not supported in TCG
> mode since it is not emulated. And only LBT feature is added here, LBT
> registers saving and restoring is not supported since it depeeds on LBT
> feautre implemented in KVM kernel

How do you test?

Thanks,

Phil.

Re: [RFC v2 1/2] target/loongarch: Add loongson binary translation feature

[maobibo]

Hi Philippe,

Thanks for reviewing my patch.
I reply inline.

On 2024/5/27 下午6:37, Philippe Mathieu-Daudé wrote:
> Hi Bibo,
> 
> On 27/5/24 10:35, Bibo Mao wrote:
>> Loongson Binary Translation (LBT) is used to accelerate binary
>> translation, which contains 4 scratch registers (scr0 to scr3), x86/ARM
>> eflags (eflags) and x87 fpu stack pointer (ftop).
>>
>> Now LBT feature is added in kvm mode, not supported in TCG mode since
>> it is not emulated. There are two feature flags such as forced_features
>> and default_features for each vcpu, the real feature is still in cpucfg.
>> Flag forced_features is parsed from command line, default_features is
>> parsed from cpu type.
>>
>> Flag forced_features has higher priority than flag default_features,
>> default_features will be used if there is no command line option for LBT
>> feature. If the feature is not supported with KVM host, it reports error
>> and exits if forced_features is set, else it disables feature and 
>> continues
>> if default_features is set.
>>
>> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
>> ---
>>   target/loongarch/cpu.c                | 69 +++++++++++++++++++++++++++
>>   target/loongarch/cpu.h                | 12 +++++
>>   target/loongarch/kvm/kvm.c            | 26 ++++++++++
>>   target/loongarch/kvm/kvm_loongarch.h  | 16 +++++++
>>   target/loongarch/loongarch-qmp-cmds.c |  2 +-
>>   5 files changed, 124 insertions(+), 1 deletion(-)
> 
> 
>> +static void loongarch_set_lbt(Object *obj, bool value, Error **errp)
>> +{
>> +    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
>> +
>> +    if (!kvm_enabled()) {
> 
> Either set errp, ...
> 
>> +        return;
>> +    }
>> +
>> +    if (value) {
>> +        /* Enable binary translation for all architectures */
>> +        cpu->env.forced_features |= BIT_ULL(LOONGARCH_FEATURE_LBT);
>> +    } else {
>> +        /* Disable default features also */
>> +        cpu->env.default_features &= ~BIT_ULL(LOONGARCH_FEATURE_LBT);
>> +    }
>> +}
>> +
>>   void loongarch_cpu_post_init(Object *obj)
>>   {
>>       object_property_add_bool(obj, "lsx", loongarch_get_lsx,
>>                                loongarch_set_lsx);
>>       object_property_add_bool(obj, "lasx", loongarch_get_lasx,
>>                                loongarch_set_lasx);
> 
> ... or only add the property if KVM is enabled:
> 
>     if (kvm_enabled()) {
Sure, will do. I think this method is better.

By the way bitmap method forced_features/default_feature is variant
of OnOffAuto method. Bitmap method uses two bit, OnOffAuto method uses 
separate feature variable. We do not know which method is better or 
which is the future trend.

Regards
Bibo Mao
> 
>> +    object_property_add_bool(obj, "lbt", loongarch_get_lbt,
>> +                             loongarch_set_lbt);
>>   }

Re: [RFC v2 0/2] target/loongarch: Add loongson binary translation feature

[maobibo]

On 2024/5/27 下午6:39, Philippe Mathieu-Daudé wrote:
> Hi Bibo,
> 
> On 27/5/24 10:34, Bibo Mao wrote:
>> Loongson Binary Translation (LBT) is used to accelerate binary
>> translation. LBT feature is added in kvm mode, not supported in TCG
>> mode since it is not emulated. And only LBT feature is added here, LBT
>> registers saving and restoring is not supported since it depeeds on LBT
>> feautre implemented in KVM kernel
> 
> How do you test?
There is a test application using LBT instruction as followings.

If LBT is not enabled, it reports illegal instruction. And it does not 
report error during VM migration.

Regards
Bibo Mao

--------------------------------------------------------------------------
#include <stdio.h>
#include <sched.h>
int main()
{
     int a = 0, b = 0;
     for (;;)
     {
         asm(
             "li.d $t0, 0xff  \n\t"
             ".word ((0x17<<18)|(0x3f<<10)|(1<<5)|0xc) \n\t" // mtflag
             ".word ((0x17<<18)|(0x3f<<10)|(0<<5)|0xc) \n\t" // mfflag
             ".word ((0x17<<18)|(0x3f<<10)|(1<<5)|0xc) \n\t" // mtflag
             "move %0, $t0 \n\t"
             : "=r"(a) : : );
         sched_yield();
         asm(
             ".word ((0x17<<18)|(0x3f<<10)|(0<<5)|0xc) \n\t" // mfflag
             "move %0, $t0 \n\t"
             : "=r"(b) : :);

         if (a != b)
         {
             printf("in: 0x%x <=> out 0x%x \n", a, b);
             return 1;
         }

         sched_yield();
         int top = 0;
         asm(
             ".word (0x8008) \n\t"                 // settm
             ".word ((0x70 << 8) | (5 << 5)) \n\t" // mttop 1
             ".word (0x8009) \n\t"                 // inctop
             : : :);
         sched_yield();
         asm(
             ".word ((0x3a0 << 5) | (0xc)) \n\t" // mfftop
             "move %0, $t0 \n\t"
             : "=r"(top) : : );

         if (top != 6)
         {
             printf("top: %d \n", top);
             return 1;
         }
     }
     return 0;
}

> 
> Thanks,
> 
> Phil.

Re: [RFC v2 1/2] target/loongarch: Add loongson binary translation feature

[gaosong]

在 2024/5/28 上午9:07, maobibo 写道:
> Hi Philippe,
>
> Thanks for reviewing my patch.
> I reply inline.
>
> On 2024/5/27 下午6:37, Philippe Mathieu-Daudé wrote:
>> Hi Bibo,
>>
>> On 27/5/24 10:35, Bibo Mao wrote:
>>> Loongson Binary Translation (LBT) is used to accelerate binary
>>> translation, which contains 4 scratch registers (scr0 to scr3), x86/ARM
>>> eflags (eflags) and x87 fpu stack pointer (ftop).
>>>
>>> Now LBT feature is added in kvm mode, not supported in TCG mode since
>>> it is not emulated. There are two feature flags such as forced_features
>>> and default_features for each vcpu, the real feature is still in 
>>> cpucfg.
>>> Flag forced_features is parsed from command line, default_features is
>>> parsed from cpu type.
>>>
>>> Flag forced_features has higher priority than flag default_features,
>>> default_features will be used if there is no command line option for 
>>> LBT
>>> feature. If the feature is not supported with KVM host, it reports 
>>> error
>>> and exits if forced_features is set, else it disables feature and 
>>> continues
>>> if default_features is set.
>>>
>>> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
>>> ---
>>>   target/loongarch/cpu.c                | 69 
>>> +++++++++++++++++++++++++++
>>>   target/loongarch/cpu.h                | 12 +++++
>>>   target/loongarch/kvm/kvm.c            | 26 ++++++++++
>>>   target/loongarch/kvm/kvm_loongarch.h  | 16 +++++++
>>>   target/loongarch/loongarch-qmp-cmds.c |  2 +-
>>>   5 files changed, 124 insertions(+), 1 deletion(-)
>>
>>
>>> +static void loongarch_set_lbt(Object *obj, bool value, Error **errp)
>>> +{
>>> +    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
>>> +
>>> +    if (!kvm_enabled()) {
>>
>> Either set errp, ...
>>
>>> +        return;
>>> +    }
>>> +
>>> +    if (value) {
>>> +        /* Enable binary translation for all architectures */
>>> +        cpu->env.forced_features |= BIT_ULL(LOONGARCH_FEATURE_LBT);
>>> +    } else {
>>> +        /* Disable default features also */
>>> +        cpu->env.default_features &= ~BIT_ULL(LOONGARCH_FEATURE_LBT);
>>> +    }
>>> +}
>>> +
>>>   void loongarch_cpu_post_init(Object *obj)
>>>   {
>>>       object_property_add_bool(obj, "lsx", loongarch_get_lsx,
>>>                                loongarch_set_lsx);
>>>       object_property_add_bool(obj, "lasx", loongarch_get_lasx,
>>>                                loongarch_set_lasx);
>>
>> ... or only add the property if KVM is enabled:
>>
>>     if (kvm_enabled()) {
> Sure, will do. I think this method is better.
>
> By the way bitmap method forced_features/default_feature is variant
> of OnOffAuto method. Bitmap method uses two bit, OnOffAuto method uses 
> separate feature variable. We do not know which method is better or 
> which is the future trend.
>
I think the OnOffAuto variable is better.

The default_features is just a copy of cpucfg, and is not required.

Thanks.
Song Gao
> Regards
> Bibo Mao
>>
>>> +    object_property_add_bool(obj, "lbt", loongarch_get_lbt,
>>> +                             loongarch_set_lbt);
>>>   }

Re: [RFC v2 1/2] target/loongarch: Add loongson binary translation feature

[maobibo]

On 2024/5/28 下午8:56, gaosong wrote:
> 在 2024/5/28 上午9:07, maobibo 写道:
>> Hi Philippe,
>>
>> Thanks for reviewing my patch.
>> I reply inline.
>>
>> On 2024/5/27 下午6:37, Philippe Mathieu-Daudé wrote:
>>> Hi Bibo,
>>>
>>> On 27/5/24 10:35, Bibo Mao wrote:
>>>> Loongson Binary Translation (LBT) is used to accelerate binary
>>>> translation, which contains 4 scratch registers (scr0 to scr3), x86/ARM
>>>> eflags (eflags) and x87 fpu stack pointer (ftop).
>>>>
>>>> Now LBT feature is added in kvm mode, not supported in TCG mode since
>>>> it is not emulated. There are two feature flags such as forced_features
>>>> and default_features for each vcpu, the real feature is still in 
>>>> cpucfg.
>>>> Flag forced_features is parsed from command line, default_features is
>>>> parsed from cpu type.
>>>>
>>>> Flag forced_features has higher priority than flag default_features,
>>>> default_features will be used if there is no command line option for 
>>>> LBT
>>>> feature. If the feature is not supported with KVM host, it reports 
>>>> error
>>>> and exits if forced_features is set, else it disables feature and 
>>>> continues
>>>> if default_features is set.
>>>>
>>>> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
>>>> ---
>>>>   target/loongarch/cpu.c                | 69 
>>>> +++++++++++++++++++++++++++
>>>>   target/loongarch/cpu.h                | 12 +++++
>>>>   target/loongarch/kvm/kvm.c            | 26 ++++++++++
>>>>   target/loongarch/kvm/kvm_loongarch.h  | 16 +++++++
>>>>   target/loongarch/loongarch-qmp-cmds.c |  2 +-
>>>>   5 files changed, 124 insertions(+), 1 deletion(-)
>>>
>>>
>>>> +static void loongarch_set_lbt(Object *obj, bool value, Error **errp)
>>>> +{
>>>> +    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
>>>> +
>>>> +    if (!kvm_enabled()) {
>>>
>>> Either set errp, ...
>>>
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    if (value) {
>>>> +        /* Enable binary translation for all architectures */
>>>> +        cpu->env.forced_features |= BIT_ULL(LOONGARCH_FEATURE_LBT);
>>>> +    } else {
>>>> +        /* Disable default features also */
>>>> +        cpu->env.default_features &= ~BIT_ULL(LOONGARCH_FEATURE_LBT);
>>>> +    }
>>>> +}
>>>> +
>>>>   void loongarch_cpu_post_init(Object *obj)
>>>>   {
>>>>       object_property_add_bool(obj, "lsx", loongarch_get_lsx,
>>>>                                loongarch_set_lsx);
>>>>       object_property_add_bool(obj, "lasx", loongarch_get_lasx,
>>>>                                loongarch_set_lasx);
>>>
>>> ... or only add the property if KVM is enabled:
>>>
>>>     if (kvm_enabled()) {
>> Sure, will do. I think this method is better.
>>
>> By the way bitmap method forced_features/default_feature is variant
>> of OnOffAuto method. Bitmap method uses two bit, OnOffAuto method uses 
>> separate feature variable. We do not know which method is better or 
>> which is the future trend.
>>
> I think the OnOffAuto variable is better.
> 
> The default_features is just a copy of cpucfg, and is not required.
No, it is not copy of cpucfg. If so, OnOffAuto is also copy of cpucfg.

Combination of forced_features/default_feature bitmap is another 
implementation of OnOffAuto variable.

   Forced_features  Default_features
      1                   x             == ON_OFF_AUTO_ON
      0                   0             == ON_OFF_AUTO_OFF
      0                   1             == ON_OFF_AUTO_AUTO

Maybe we can rename Default_features with Hint_features.

Regards
Bibo Mao

> 
> Thanks.
> Song Gao
>> Regards
>> Bibo Mao
>>>
>>>> +    object_property_add_bool(obj, "lbt", loongarch_get_lbt,
>>>> +                             loongarch_set_lbt);
>>>>   }
>