Customizing dialog by extending Dialog or AlertDia

I want to make a custom Dialog. Because I don't like its style, I want to have rounded rectangle rather than sharp corners. I know how to implement it by theme in AndroidManifest.xml, for example, I use:

android:theme="@style/Theme.CustomDialog"

And Theme.CustomDialog.xml:

<style name="Theme.CustomDialog" parent="android:style/Theme.Dialog">
        <item name="android:windowBackground">@drawable/filled_box</item>
        <item name="android:windowNoTitle">true</item>

filled_box.xml
<shape xmlns:android="http://schemas.android.com/apk/res/android">
    <solid android:color="#ffffffff"/>
    <stroke android:width="3dp" color="#ffff8080"/>
    <corners android:radius="30dp" />
    <padding android:left="10dp" android:top="10dp"
        android:right="10dp" android:bottom="10dp" />
</shape>

How can I implement a similar result by extending the Dialog or AlertDialog?

In the constructor of your class that extends Dialog call super(context, R.style.CustomDialog); I've done this many times to create custom dialogs with specific themes.

However if the theme is the only thing about the Dialog that you want to change, you could try just instantiating an instance of the Dialog class and pass it the theme ID like Dialog dialog = new Dialog(context, R.style.CustomDialog);

An example of extending Dialog:

public class MyDialog extends Dialog
{
    public MyDialog(final Context context)
    {
        // Set your theme here
        super(context, R.style.MyDialogTheme);

        // This is the layout XML file that describes your Dialog layout
        this.setContentView(R.layout.myDialogLayout);  
    }
}

The rest of the code you will add to this class is going to be pretty much exactly like what you would write in an Activity class.