Most people delete their files by selecting them and pressing the Delete key, or by dragging them into the Recycle Bin. These files will be stored in the Recycle Bin until you right-click it and select 'Empty Recycle Bin' command.
To force these files to be deleted immediately, hold down the Shift key while deleting the files. This will cause the files that your currently deleting to be purged without first moving them to the Recycle Bin.
Note: Any files that are currently stored in the Recycle Bin will not be deleted when using this method.