EJBContext is an interface that is implemented by the container, and it is also a part of the bean-container contract. Entity beans use a subclass of EJBContext called EntityContext. Session beans use a subclass called SessionContext. These EJBContext objects provide the bean class with information about its container, the client using the bean and the bean itself. They also provide other functions. See the API docs and the spec for more details.
EJBs can be clients of other EJBs. It just works. Use JNDI to locate the Home Interface of the other bean, then acquire an instance reference, and so forth.
An Entity Bean represents persistent data that is stored outside of the EJB Container/Server. The ejbFindByPrimaryKey is a method used to locate and load an Entity Bean into the container, similar to a SELECT statement in SQL. By making this method mandatory, the client programmer can be assured that if they have the primary key of the Entity Bean, then they can retrieve the bean without having to create a new bean each time - which would mean creating duplications of persistent data and break the integrity of EJB.
With the EJBHome version of the remove, you are able to delete an entity bean without first instantiating it (you can provide a PrimaryKey object as a parameter to the remove method). The home version only works for entity beans. On the other hand, the Remote interface version works on an entity bean that you have already instantiated. In addition, the remote version also works on session beans (stateless and stateful) to inform the container of your loss of interest in this bean.
An EJB server is an application, usually a product such as BEA WebLogic, that provides (or should provide) for concurrent client connections and manages system resources such as threads, processes, memory, database connections, network connections, etc. An EJB container runs inside (or within) an EJB server, and provides deployed EJB beans with transaction and security management, etc. The EJB container insulates an EJB bean from the specifics of an underlying EJB server by providing a simple, standard API between the EJB bean and its container. A Connector provides the ability for any Enterprise Information System (EIS) to plug into any EJB server which supports the Connector architecture.